Code World Model Benchmark

A coding agent photographs a black-box physics scene under a budget, never sees its source, writes a Blender or Genesis script that reproduces it, and is scored from five secret viewpoints it never saw.

How the exam works

Each of the 200 tasks is a black-box physics scene, 100 keyframed Blender scenes (b001–b100) and 100 rigid-body Genesis machines (g001–g100), that the agent can only observe by requesting renders from viewpoints of its choosing. Under photos the budget is 3 viewpoints × 4 frames and 5 test renders; under open the agent may request as many viewpoints, frames and test renders as it wants, within a hard cap of 40 renders and 60 minutes per task. The agent submits one script; the evaluator renders the original and the submission from five secret viewpoints, seeded per task and never shown to the agent, and compares them frame by frame with LPIPS (DreamSim, SSIM and DINO are also reported). A task passes when the mean LPIPS over the five secret views is ≤ 0.02, a threshold calibrated on the noise floor of re-rendering the original; pass@200 is passes divided by 200, and a task the agent observed but never submitted counts as a failure.

hokinxqdeng@gmail.com