BuilderBuilder

From code, to simulation, to physical machines.

Independent AI benchmark lab. Same task, frozen prompt, every failure kept.

· Latest test · Launch 006

Opus 5.5 vs GPT-6 Astra vs Sonnet 5.5 vs GPT-6.1 Sol

Four models across games, visual scenes, CAD, PCB, and simulated robotics, with per-round evidence and no aggregate score. Unaided model attempts stay separate from Astra planner + Jev footage. Ledger status is not replaced by the narration. Robot clips are simulated.

See the comparison Watch the reviewInspect all public resultsExplore benchmark projects and reproduction paths
Claude Opus 5.5 · Draw the Eiffel Tower
Claude Opus 5.5 · Draw the Eiffel Tower

Experiments

All experiments →
  1. Opus 5.5 vs GPT-6 Astra vs Sonnet 5.5 vs GPT-6.1 SolNo aggregate score. Each round keeps its own ledger outcome.Games · Graphics · CAD/3D · Interactive worlds · Robot simulation · 20 tests
  2. Qwen 3.8 Flash vs DeepSeek V4.1 Flash vs GLM 5.3 Flash vs Gemini 3.8 FlashThere was no universal winner. Flash-model performance was brutally task-specific.Games · Graphics · CAD/3D · Robot simulation · 14 tests
  3. Claude Opus 5 across nine game-building testsOpus 5 delivered the strongest detail in several rounds. It also made the efficiency argument against itself.Games · Interactive worlds · 9 tests
  4. I Gave Qwen3.8 Max, Kimi K3, GPT-5.6 and Fable 5 the Same 9 Game TestsKimi K3 was more consistent; Qwen's visual-agent workflow was not reliable enough to win overall.Games · Interactive worlds · 9 tests
  5. Kimi K3 vs GPT-5.6 SOL vs Claude Fable 5: 9 Game TestsFable 5 sets the visual ceiling. Kimi K3 wins on value. GPT-5.6 Sol Ultra is the practical middle when technical coherence matters without paying Fable’s full cost.Games · Interactive worlds · 9 tests
  6. GPT-5.6 vs Claude Fable 5: Same Prompts, 7 AI Game-Dev TestsFable sets the visual ceiling. Sol Ultra wins the value argument.Games · Interactive worlds · 7 tests

Models

All results →
ModelProviderRunsAvg costAvg time
GLM 5.3 FlashOpenCode Go34$1.23 · includes provider estimates1h 14m
Claude Opus 5.5Anthropic28$48.0629m 51s
GPT-6 AstraExisting Doom simulation28$31.941h 11m
GPT-5.6 LunaOpenAI Codex27$2.0323m 26s
GPT-5.6 SolOpenAI Codex26$8.4539m 15s
Grok 4.6Grok CLI subscription26$4.85 · includes provider estimates33m 25s
DeepSeek V4 FlashOpenCode Go25$0.261h 56m
Claude Opus 5Anthropic23≥$68.522h 31m

Method

Methodology →
  1. Per roundEach round names its own target and configuration. Doom and ATS control existing simulations. Efforts and interventions are not one shared setup.
  2. Disclosed exceptionsLaunch 006 keeps retained retries, quota stops, and harness repairs on the round. Astra planner + Jev is a separate workflow.
  3. Failures retainedFailed and partial runs stay in the record.
  4. No run, no scoreUntested systems stay untested — including physical machines.