# // toolbench — ai tools building benchmark
[ updated: Jul 2026 ] [ models: 5 ] [ problems: 5 ] [ dimensions: 4 ]
## // methodology
✓
Correctness

Output matches spec exactly. Handles edge cases, error states, and unexpected inputs without crashing.

◉
Efficiency

Tool outputs well-optimized solutions. Minimal dependencies, proper caching, appropriate algorithms.

◎
Composability

Generated artifacts compose well: modular, testable, importable. Clear interfaces between components.

✦
Robustness

Error handling, input validation, logging, and graceful degradation. Production-ready quality.

How scores work: Each model is given the same 5 tool-building problems at temperature 0.2. Solutions are evaluated programmatically — no human judgment involved. Composite = correctness × 0.4 + efficiency × 0.2 + composability × 0.2 + robustness × 0.2. New models added as they ship tool-building capabilities.
## // overall standings
#1
Nexum Router (Dialagram)
0.0
#2
Claude Code (Anthropic)
0.0
#3
Codex CLI (OpenAI)
0.0
#4
Aider (Aider AI)
0.0
#5
Cursor Agent (Cursor)
0.0
## // per-model breakdown

Nexum Router (Dialagram)

dialagram · nexum-router · Overall: 0.0
✓
0.0
Correctness
◉
0.0
Efficiency
◎
0.0
Composability
✦
0.0
Robustness
∑
Git Bisect
0.0
F
Refactor Import
0.0
⟐
Build CLI Tool
0.0
⚡
CI Pipeline
0.0
{}
API Wrapper
0.0

Claude Code (Anthropic)

anthropic · claude-code · Overall: 0.0
✓
0.0
Correctness
◉
0.0
Efficiency
◎
0.0
Composability
✦
0.0
Robustness
∑
Git Bisect
0.0
F
Refactor Import
0.0
⟐
Build CLI Tool
0.0
⚡
CI Pipeline
0.0
{}
API Wrapper
0.0

Codex CLI (OpenAI)

openai · codex-cli · Overall: 0.0
✓
0.0
Correctness
◉
0.0
Efficiency
◎
0.0
Composability
✦
0.0
Robustness
∑
Git Bisect
0.0
F
Refactor Import
0.0
⟐
Build CLI Tool
0.0
⚡
CI Pipeline
0.0
{}
API Wrapper
0.0

Aider (Aider AI)

aider · aider · Overall: 0.0
✓
0.0
Correctness
◉
0.0
Efficiency
◎
0.0
Composability
✦
0.0
Robustness
∑
Git Bisect
0.0
F
Refactor Import
0.0
⟐
Build CLI Tool
0.0
⚡
CI Pipeline
0.0
{}
API Wrapper
0.0

Cursor Agent (Cursor)

cursor · cursor-agent · Overall: 0.0
✓
0.0
Correctness
◉
0.0
Efficiency
◎
0.0
Composability
✦
0.0
Robustness
∑
Git Bisect
0.0
F
Refactor Import
0.0
⟐
Build CLI Tool
0.0
⚡
CI Pipeline
0.0
{}
API Wrapper
0.0
## // per-problem comparison

∑ Git Bisect

Find the exact commit that introduced a regression in a 500-commit repo using git bisect automation.

Model ✓ Correctness◉ Efficiency◎ Composability✦ Robustness Composite
Nexum Router (Dialagram) 0.0 0.0 0.0 0.0
0.0
Claude Code (Anthropic) 0.0 0.0 0.0 0.0
0.0
Codex CLI (OpenAI) 0.0 0.0 0.0 0.0
0.0
Aider (Aider AI) 0.0 0.0 0.0 0.0
0.0
Cursor Agent (Cursor) 0.0 0.0 0.0 0.0
0.0

F Refactor Import

Refactor a 200-line monolith into a modular package with proper imports, type stubs, and init files.

Model ✓ Correctness◉ Efficiency◎ Composability✦ Robustness Composite
Nexum Router (Dialagram) 0.0 0.0 0.0 0.0
0.0
Claude Code (Anthropic) 0.0 0.0 0.0 0.0
0.0
Codex CLI (OpenAI) 0.0 0.0 0.0 0.0
0.0
Aider (Aider AI) 0.0 0.0 0.0 0.0
0.0
Cursor Agent (Cursor) 0.0 0.0 0.0 0.0
0.0

⟐ Build CLI Tool

Build a CLI tool from spec: parse flags, handle errors, output JSON/table, with subcommands.

Model ✓ Correctness◉ Efficiency◎ Composability✦ Robustness Composite
Nexum Router (Dialagram) 0.0 0.0 0.0 0.0
0.0
Claude Code (Anthropic) 0.0 0.0 0.0 0.0
0.0
Codex CLI (OpenAI) 0.0 0.0 0.0 0.0
0.0
Aider (Aider AI) 0.0 0.0 0.0 0.0
0.0
Cursor Agent (Cursor) 0.0 0.0 0.0 0.0
0.0

⚡ CI Pipeline

Write a multi-stage CI pipeline config (lint, test, build, deploy) with caching and parallel jobs.

Model ✓ Correctness◉ Efficiency◎ Composability✦ Robustness Composite
Nexum Router (Dialagram) 0.0 0.0 0.0 0.0
0.0
Claude Code (Anthropic) 0.0 0.0 0.0 0.0
0.0
Codex CLI (OpenAI) 0.0 0.0 0.0 0.0
0.0
Aider (Aider AI) 0.0 0.0 0.0 0.0
0.0
Cursor Agent (Cursor) 0.0 0.0 0.0 0.0
0.0

{} API Wrapper

Build a type-safe API client wrapper for a REST service with auth, retry, and pagination.

Model ✓ Correctness◉ Efficiency◎ Composability✦ Robustness Composite
Nexum Router (Dialagram) 0.0 0.0 0.0 0.0
0.0
Claude Code (Anthropic) 0.0 0.0 0.0 0.0
0.0
Codex CLI (OpenAI) 0.0 0.0 0.0 0.0
0.0
Aider (Aider AI) 0.0 0.0 0.0 0.0
0.0
Cursor Agent (Cursor) 0.0 0.0 0.0 0.0
0.0
## // key takeaways

Placeholder — real scores coming soon

This page is a scaffold. Actual benchmark runs will populate scores as we test each model against the tool-building problem set. Check back for real data.

Tool building is the new frontier

Unlike code generation benchmarks (LeetCode, HumanEval), tool-building tests measure a model's ability to produce production-grade, composable artifacts with real-world constraints.

We test what matters

Correctness isn't enough — a tool that passes tests but fails on error handling or composability isn't production-ready. Our 4-dimensional scoring surfaces these trade-offs.

Open methodology

Every problem spec, test harness, and scoring script is open source. Run the benchmarks yourself or suggest new problems via GitHub.