Bitbench.org

Bitcoin AI Benchmark

Results About

BSVv1.1.0Last run Dec 18, 2025, 10:58 PM

BSV Data Protocols

Tests parsing of MAP, B, AIP, Sigma, BAP, and Run protocols from raw hex.

Results

2 runs, Dec 18, 2025, 10:44 PM to Dec 18, 2025, 10:58 PM. Each model shows its own run date.

Archived results from v2025-12-18. The questions have since changed in v1.1.0.Back to v1.1.0

Success Rate by Model

Percentage of correct answers out of 6 tests per model

Funding

$0.00 raised$13.39 goal

$13.39 left to reach the goal. Results publish automatically once funded and run.

Donation address

Models

141 models run against this suite, the full current-model registry.

Show all models

claude-sonnet-5

claude-fable-5

claude-opus-4.8-fast

claude-opus-4.8

claude-opus-4.7-fast

claude-opus-4.7

claude-sonnet-4.6

claude-opus-4.6

claude-4.5-opusDec 18, 2025, 10:58 PM

claude-4.5-opus-thinking-highDec 18, 2025, 10:58 PM

claude-haiku-4.5

claude-sonnet-4.5

claude-opus-4.1

gpt-5.6-luna-pro

gpt-5.6-luna

gpt-5.6-terra-pro

gpt-5.6-terra

gpt-5.6-sol-pro

gpt-5.6-sol

gpt-chat-latest

gpt-5.5-pro

gpt-5.5

gpt-5.4-nano

gpt-5.4-mini

gpt-5.4-pro

gpt-5.4

gpt-5.3-chat

gpt-5.3-codex

gpt-5.2-codex

gpt-5.2-chat

gpt-5.2-proDec 18, 2025, 10:44 PM

gpt-5.2-noneDec 18, 2025, 10:58 PM

gpt-5.2-xhigh

gpt-5.2-highDec 18, 2025, 10:58 PM

gpt-5.1-codex-max

gpt-5.1-lowDec 18, 2025, 10:58 PM

gpt-5.1-defaultDec 18, 2025, 10:58 PM

gpt-5.1-highDec 18, 2025, 10:58 PM

gpt-5.1-chat

gpt-5.1-codex

gpt-5.1-codex-mini

gpt-oss-safeguard-20b

o3-deep-research

o4-mini-deep-research

gpt-5-pro

gpt-5-codex

gpt-5-chat

gpt-5-minimalDec 18, 2025, 10:58 PM

gpt-5-defaultDec 18, 2025, 10:58 PM

gpt-5-highDec 18, 2025, 10:58 PM

gpt-5-miniDec 18, 2025, 10:58 PM

gpt-5-nanoDec 18, 2025, 10:58 PM

gpt-oss-120bDec 18, 2025, 10:58 PM

gpt-oss-20bDec 18, 2025, 10:58 PM

gemini-3.5-flash

gemini-3.1-flash-lite

gemma-4-26b-a4b-it

gemma-4-31b-it

gemini-3.1-flash-lite-preview

gemini-3.1-pro-preview-customtools

gemini-3.1-pro-preview

gemini-3-flash-highDec 18, 2025, 10:58 PM

gemini-3-flash-lowDec 18, 2025, 10:58 PM

gemini-2.5-flash-lite

grok-4.5

grok-build-0.1

grok-4.3

grok-4.20-multi-agent

grok-4.20

deepseek-v4-pro

deepseek-v4-flash

deepseek-v3.2Dec 18, 2025, 10:58 PM

deepseek-v3.2-thinking-highDec 18, 2025, 10:58 PM

deepseek-v3.2-exp

deepseek-v3.1-terminus

deepseek-v3.1Dec 18, 2025, 10:58 PM

deepseek-v3.1-thinkingDec 18, 2025, 10:58 PM

qwen3.7-plus

qwen3.7-max

qwen3.5-plus-20260420

qwen3.6-flash

qwen3.6-35b-a3b

qwen3.6-max-preview

qwen3.6-27b

qwen3.6-plus

qwen3.5-9b

qwen3.5-35b-a3b

qwen3.5-27b

qwen3.5-122b-a10b

qwen3.5-flash-02-23

qwen3.5-plus-02-15

qwen3.5-397b-a17b

qwen3-max-thinking

qwen3-coder-next

qwen3-vl-32b-instruct

qwen3-vl-8b-thinking

qwen3-vl-8b-instruct

qwen3-vl-30b-a3b-thinking

qwen3-vl-30b-a3b-instruct

qwen3-vl-235b-a22b-thinking

qwen3-vl-235b-a22b-instruct

qwen3-max

qwen3-coder-plus

qwen3-coder-flash

qwen3-next-80b-a3b-thinking

qwen3-next-80b-a3b-instruct

qwen-plus-2025-07-28

qwen-plus-2025-07-28:thinking

qwen3-30b-a3b-thinking-2507

qwen3-coder-30b-a3b-instruct

qwen3-30b-a3b-instruct-2507

qwen3-235b-a22b-thinking-2507

qwen3-coder

qwen3-235b-a22b-2507

kimi-k2.7-code

kimi-k2.6

kimi-k2.5

kimi-k2-thinkingDec 18, 2025, 10:58 PM

kimi-k2-0905

mistral-medium-3-5

mistral-small-2603

devstral-2512

ministral-14b-2512

ministral-8b-2512

ministral-3b-2512

mistral-large-2512

voxtral-small-24b-2507

mistral-medium-3.1

codestral-2508

glm-5.2

glm-5.1

glm-5v-turbo

glm-5-turbo

glm-5

glm-4.7-flash

glm-4.7

glm-4.6v

glm-4.6

glm-4.5vDec 18, 2025, 10:58 PM

glm-4.5Dec 18, 2025, 10:58 PM

glm-4.5-air

Questions

version 2025-12-18

Full item bank in the repository