Per-provider performance
Latency, throughput, price, and reliability per (model, provider). Quality is judged once on the cheapest provider — perf is measured independently so the leaderboard isn't biased by speed or price. legacy rows come from earlier OpenRouter sweeps (TTFT + total ms); phase3 rows are the new harness (full latency + observed $/Mtok).
Normal-play cost profile
Cost estimates use 14.10 gameplay request(s)/minute,
about 12378.46 input token(s)/minute and
705.01 output token(s)/minute.
The synthetic demo-player category is excluded. Source: docs/proofs/demo-api-profile/20260524T021229Z/summary.json.
| Category | req/min | input tok/req | output tok/req | input tok/min | output tok/min |
|---|---|---|---|---|---|
| dialogue | 1.47 | 2289.57 | 158.14 | 3373.92 | 233.04 |
| intent | 2.53 | 496.83 | 34.33 | 1255.09 | 86.73 |
| reaction | 4.00 | 550.89 | 10.21 | 2203.46 | 40.84 |
| simulation | 5.68 | 955.07 | 58.22 | 5428.54 | 330.93 |
| travel | 0.42 | 279.00 | 32.00 | 117.47 | 13.47 |
| Total gameplay | 14.10 | 877.63 | 49.99 | 12378.46 | 705.01 |
Catalog cloud cost examples
Examples combine the same normal-play profile with input/output prices from
rundale-bench/v1/models.toml. Prices are point-in-time estimates; verify provider pricing before spending money. Click a column header to sort.
| Model | Provider | input $/M | output $/M | game $/min | game $/hr ▲ |
|---|---|---|---|---|---|
| Llama-3.3 70B Instruct | openrouter | $0.13 | $0.39 | $0.0019 | $0.11 |
| DeepSeek V4 Flash | opencode-go | $0.14 | $0.28 | $0.0019 | $0.12 |
| Grok 4 Fast | xai | $0.20 | $0.50 | $0.0028 | $0.17 |
| Qwen3.5 Plus | opencode-go | $0.20 | $1.20 | $0.0033 | $0.20 |
| DeepSeek V3 | deepseek | $0.27 | $1.10 | $0.0041 | $0.25 |
| DeepSeek V3 | openrouter | $0.28 | $1.10 | $0.0042 | $0.25 |
| GGPT-5 mini | openai | $0.25 | $2.00 | $0.0045 | $0.27 |
| MiniMax M2.5 | opencode-go | $0.30 | $1.20 | $0.0046 | $0.27 |
| MiniMax M2.7 | opencode-go | $0.30 | $1.20 | $0.0046 | $0.27 |
| MiMo V2.5 | opencode-go | $0.40 | $2.00 | $0.0064 | $0.38 |
| Llama-3.3 70B Instruct | groq | $0.59 | $0.79 | $0.0079 | $0.47 |
| Qwen3.6 Plus | opencode-go | $0.50 | $3.00 | $0.0083 | $0.50 |
| Kimi K2.5 | opencode-go | $0.60 | $3.00 | $0.0095 | $0.57 |
| Llama-3.3 70B Instruct | together | $0.88 | $0.88 | $0.0115 | $0.69 |
| MiMo V2.5 Pro | opencode-go | $1.00 | $3.00 | $0.0145 | $0.87 |
| Kimi K2.6 | opencode-go | $0.95 | $4.00 | $0.0146 | $0.87 |
| GGLM-5 | opencode-go | $1.00 | $3.20 | $0.0146 | $0.88 |
| Claude Haiku 4.5 | anthropic | $1.00 | $5.00 | $0.0159 | $0.95 |
| GGLM-5.1 | opencode-go | $1.40 | $4.40 | $0.0204 | $1.23 |
| DeepSeek V4 Pro | opencode-go | $1.74 | $3.48 | $0.0240 | $1.44 |
| Model | Provider | total p50 ms ▲ | total p95/p90 ms | tok/s | ttft p50 ms | $/Mtok | game $/min | game $/hr | err | source |
|---|---|---|---|---|---|---|---|---|---|---|
| mistralai/mistral-small-24b-instruct-2501 | openrouter | 838 | — | 77.4 | 250 | — | — | — | 0% | legacy |
| amazon/nova-pro-v1 | openrouter | 855 | — | 65.9 | 478 | — | — | — | 0% | legacy |
| google/gemini-2.5-flash | openrouter | 1032 | — | 99.5 | 595 | — | — | — | 0% | legacy |
| openai/gpt-4o-mini | openrouter | 1243 | — | 192.7 | 816 | — | — | — | 0% | legacy |
| openai/gpt-5.4-mini | openrouter | 1389 | — | 75.5 | 561 | — | — | — | 0% | legacy |
| meta-llama/llama-4-scout | openrouter | 1419 | — | 54.2 | 279 | — | — | — | 0% | legacy |
| meta-llama/llama-4-maverick | openrouter | 1601 | — | 51.1 | 412 | — | — | — | 0% | legacy |
| microsoft/phi-4 | openrouter | 1674 | — | 67.8 | 364 | — | — | — | 0% | legacy |
| qwen/qwen3-235b-a22b-2507 | openrouter | 1696 | — | 54.2 | 363 | — | — | — | 0% | legacy |
| mistralai/mistral-large-2512 | openrouter | 1714 | — | 44.1 | 482 | — | — | — | 0% | legacy |
| anthropic/claude-haiku-4.5 | openrouter | 1989 | — | 81.6 | 980 | — | — | — | 0% | legacy |
| mistralai/mistral-medium-3.1 | openrouter | 2089 | — | 50.4 | 390 | — | — | — | 0% | legacy |
| google/gemma-3-27b-it | openrouter | 2328 | — | 37.9 | 380 | — | — | — | 0% | legacy |
| meta-llama/llama-3.3-70b-instruct | openrouter | 2339 | — | 43.2 | 516 | — | $0.0019 | $0.11 | 0% | legacy |
| openai/gpt-5.4 | openrouter | 2677 | — | 34.7 | 715 | — | — | — | 0% | legacy |
| deepseek/deepseek-v3.2 | openrouter | 2704 | — | 22.2 | 691 | — | — | — | 0% | legacy |
| minimax-m2.7 | opencode-go | 2781 | — | 4975.4 | 2741 | — | $0.0046 | $0.27 | 0% | legacy |
| qwen/qwen3-max | openrouter | 2904 | — | 32.6 | 1173 | — | — | — | 0% | legacy |
| openai/gpt-oss-120b | openrouter | 3072 | — | 62.7 | 427 | — | — | — | 0% | legacy |
| x-ai/grok-4.3 | openrouter | 3183 | — | 207.5 | 704 | — | — | — | 0% | legacy |
| google/gemma-4-31b-it | openrouter | 3214 | — | 21.9 | 1160 | — | — | — | 0% | legacy |
| anthropic/claude-sonnet-4.6 | openrouter | 3221 | — | 44.1 | 1328 | — | — | — | 0% | legacy |
| google/gemini-2.5-pro | openrouter | 3331 | — | 143.3 | 1925 | — | — | — | 0% | legacy |
| deepseek-v4-flash | opencode-go | 3799 | — | — | — | — | $0.0019 | $0.12 | 0% | legacy |
| anthropic/claude-opus-4.7 | openrouter | 3861 | — | 39.0 | 1421 | — | — | — | 0% | legacy |
| openai/gpt-5.5 | openrouter | 4067 | — | 60.0 | 2242 | — | — | — | 0% | legacy |
| mimo-v2.5 | opencode-go | 4196 | — | 469.6 | 3670 | — | $0.0064 | $0.38 | 0% | legacy |
| qwen/qwen-2.5-72b-instruct | openrouter | 5242 | — | — | — | — | — | — | 90% | legacy |
| minimax-m2.5 | opencode-go | 5504 | — | 43.3 | 1392 | — | $0.0046 | $0.27 | 0% | legacy |
| nousresearch/hermes-4-405b | openrouter | 5623 | — | 39.4 | 333 | — | — | — | 0% | legacy |
| deepseek/deepseek-v4-pro | openrouter | 6170 | — | 39.2 | 1317 | — | — | — | 0% | legacy |
| deepseek-v4-pro | opencode-go | 6335 | — | — | — | — | $0.0240 | $1.44 | 0% | legacy |
| kimi-k2.5 | opencode-go | 6759 | — | — | — | — | $0.0095 | $0.57 | 0% | legacy |
| moonshotai/kimi-k2.5 | openrouter | 7220 | — | 37.6 | 942 | — | — | — | 0% | legacy |
| kimi-k2.6 | opencode-go | 7269 | — | — | — | — | $0.0146 | $0.87 | 0% | legacy |
| Gglm-5.1 | opencode-go | 7571 | — | 279.7 | 6225 | — | $0.0204 | $1.23 | 0% | legacy |
| mimo-v2.5-pro | opencode-go | 7714 | — | 354.6 | 6314 | — | $0.0145 | $0.87 | 0% | legacy |
| Gglm-5 | opencode-go | 7848 | — | 167.4 | 6657 | — | $0.0146 | $0.88 | 0% | legacy |
| z-ai/glm-4.6 | openrouter | 16883 | — | 34.7 | 1442 | — | — | — | 0% | legacy |
| qwen3.6-plus | opencode-go | 55809 | — | 961.3 | 52666 | — | $0.0083 | $0.50 | 0% | legacy |
| qwen3.5-plus | opencode-go | 81710 | — | 2081.4 | 79646 | — | $0.0033 | $0.20 | 50% | legacy |
| x-ai/grok-3-mini | openrouter | — | — | — | — | — | — | — | 100% | legacy |