Skip to content
Model Pareto

Compare models

Side by side: quality per category, price, speed, and every benchmark cell with where it came from. Pick up to four.
GPT-6.1 SolClaude Opus 5.5Granite 4.2 3B
Focus

Coding: quality vs price

Compared models are ringed; the other 102 ranked here are greyed.

Lower price is better. Pareto frontier: Granite 4.2 3B, NVIDIA Nemotron 3 Nano 30B A3B, gpt-oss-20b, Ling-3.0-flash-VL, MiMo-V2.6-Flash, GLM-5.3 Flash, DeepSeek V4.1 Flash, MiMo-V2.6-Pro, Claude Sonnet 5.5, Claude Opus 5.5. 27 models are not plotted: Xing4.0-29B-A4B, Nex-N2.5-Pro, Atria Dawn Preview, Intern-S2-397B, Solar Open2 250B, Phi-4-reasoning-plus, Motif 3, Nex-N2-Pro, Nex-N2.5-Mini, A.X-K2, K2 Horizon 375B A23B, K-EXAONE 2.0 0803, K2 Horizon MoVA 36B A4B, Intern-S2-Preview (35B-A3B), Command A+, North Mini Code, G9v3-39A5B, Falcon-H1-34B-Instruct, Devstral 2, K2 Horizon 7B, K2-Horizon-32B, Devstral Small 2, MiniCPM5-2B, Ling 3.0 Tiny, K2 Horizon 3.7B, K2 Horizon 0.9B, LFM2.5-2.6B.

27 models with no price data — shown in the strip at the left edge

Xing4.0-29B-A4B, Nex-N2.5-Pro, Atria Dawn Preview, Intern-S2-397B, Solar Open2 250B, Phi-4-reasoning-plus, Motif 3, Nex-N2-Pro, Nex-N2.5-Mini, A.X-K2, K2 Horizon 375B A23B, K-EXAONE 2.0 0803, K2 Horizon MoVA 36B A4B, Intern-S2-Preview (35B-A3B), Command A+, North Mini Code, G9v3-39A5B, Falcon-H1-34B-Instruct, Devstral 2, K2 Horizon 7B, K2-Horizon-32B, Devstral Small 2, MiniCPM5-2B, Ling 3.0 Tiny, K2 Horizon 3.7B, K2 Horizon 0.9B, LFM2.5-2.6B

  • Best-value frontier (nothing is both cheaper and better)
  • Evidencestrong → weak
  • Estimated from other categories
  • GPT-6.1 Sol
    Quality
    85.3
    Rank
    #5/105
    Price
    $4.00/M tok
    Speed
    —
    In$2.00Out$10/M tok

    vs Claude Opus 5.5: −11.4 quality · 0.5× price

  • Claude Opus 5.5
    Quality
    96.6
    Rank
    #1/105
    Price
    $8.00/M tok
    Speed
    92tok/s
    In$4.00Out$20/M tok

    vs GPT-6.1 Sol: +11.4 quality · 2× price

  • Granite 4.2 3B
    Quality
    17.4
    Rank
    #97/105
    Price
    $0.05/M tok
    Speed
    220tok/s
    In$0.03▾Out$0.12▾/M tok

    vs Claude Opus 5.5: −79.2 quality · 0.01× price

Overview

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
Lab
OpenAI
Anthropic
IBM Granite
Released
Sep 29, 2026
Sep 22, 2026
Aug 25, 2026
Weights
Proprietary
Proprietary
Open
Input price
$/M tok
$2.00
$4.00
$0.03Best
Output price
$/M tok
$10
$20
$0.12Best
Blended price
$/M tok · 3:1 in:out
$4.00
$8.00
$0.052Best
Output speed
tok/s
—
92
220Best
Context
tokens
922K
1MBest
131K

Quality by category

0–100 within each category (100 = best tracked model). Rank is among all models ranked there. Click a row to focus it.

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
Coding
85.3±6.4
#5/105Mixed
96.6±3.5Best
#1/105Mixedbest value
17.4±8.0
#97/105Verifiedbest value
Writing & chat
92.6±10.0
#4/147Estimated from other categoriesbest value
93.0±4.3Best
#3/147Verifiedbest value
Not ranked
Vision
85.1±8.2
#5/83Verified
94.5±4.8Best
#2/83Verifiedbest value
Not ranked
Hard reasoning
91.5±5.1
#6/184Verified
99.9±2.6Best
#1/184Verifiedbest value
23.3±3.7
#129/184Verified
Agents
84.7±12.5
#9/175Estimated from other categories
99.6±4.1Best
#1/175Verifiedbest value
17.1±5.8
#142/175Verified

Coding benchmarks

Rank among models with a published score, and the leaderboard around each model. ~ italic = no published score, estimated from related benchmarks.

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
SWE-bench Verified
day 0
~#2/14~90.2%Estimated
  1. #1Claude Opus 596.0%
  2. ~#2GPT-6.1 SolEstimated90.2%
  3. #2Claude Sonnet 585.2%
estimated from related benchmarks
~#2/14~94.7%Estimated
  1. #1Claude Opus 596.0%
  2. ~#2Claude Opus 5.5Estimated94.7%
  3. #2Claude Sonnet 585.2%
estimated from related benchmarks
~#12/14~64.6%Estimated
  1. #1Claude Opus 596.0%
  2. #11Nemotron 3 Ultra70.7%
  3. ~#12Granite 4.2 3BEstimated64.6%
  4. #12Intern-S2-Preview (35B-A3B)64.0%
estimated from related benchmarks
SWE-bench Pro (Public)
day 0
~#5/19~75.8%Estimated
  1. #1Claude Opus 5.589.9%
  2. #4Claude Opus 579.2%
  3. ~#5GPT-6.1 SolEstimated75.8%
  4. #5Intern-S2-397B68.5%
estimated from related benchmarks
#1/1889.9%Lab-reported
  1. #1Claude Opus 5.589.9%
  2. #2Claude Sonnet 5.581.3%
www-cdn.anthropic.com · 2026-09-22
~#18/19~45.8%Estimated
  1. #1Claude Opus 5.589.9%
  2. #17Muse Glimmer51.2%
  3. ~#18Granite 4.2 3BEstimated45.8%
  4. #18Nex-N2.5-Mini43.8%
estimated from related benchmarks
DeepSWE v1.1
day 0
#1/3175.2%BestLab-reported
  1. #1GPT-6.1 Sol75.2%
  2. #2DeepSeek V4.1 Flash74.2%
vellum.ai · 2026-09-29
#9/3168.4%Verified
  1. #1GPT-6.1 Sol75.2%
  2. #8GPT-6 Sol69.0%
  3. #9Claude Opus 5.568.4%
  4. #9Kimi K368.4%
artificialanalysis.ai · 2026-09-24
~#30/32~38.2%Estimated
  1. #1GPT-6.1 Sol75.2%
  2. #29Qwen3.8 27B42.2%
  3. ~#30Granite 4.2 3BEstimated38.2%
  4. #30Nex-N2.5-Mini36.1%
estimated from related benchmarks
Terminal-Bench 4.0
day 0
#4/8456.1%Verified
  1. #1Claude Sonnet 5.563.6%
  2. #3GPT-6 Astra59.1%
  3. #4GPT-6.1 Sol56.1%
  4. #5Claude Fable 5.152.0%
artificialanalysis.ai · 2026-09-29
#2/8459.6%BestVerified
  1. #1Claude Sonnet 5.563.6%
  2. #2Claude Opus 5.559.6%
  3. #3GPT-6 Astra59.1%
artificialanalysis.ai · 2026-09-24
#50/840.0%Verified
  1. #1Claude Sonnet 5.563.6%
  2. #50gpt-oss-20b0.0%
  3. #50Granite 4.2 3B0.0%
  4. #50Granite 4.2 8B0.0%
artificialanalysis.ai · 2026-09-24
LiveCodeBench
third-party
~#2/20~92.1%Estimated
  1. #1Qwen3.8-Omni-Flash92.6%
  2. ~#2GPT-6.1 SolEstimated92.1%
  3. #2Qwen3.8 Flash-Next91.9%
estimated from related benchmarks
~#2/20~92.3%Estimated
  1. #1Qwen3.8-Omni-Flash92.6%
  2. ~#2Claude Opus 5.5Estimated92.3%
  3. #2Qwen3.8 Flash-Next91.9%
estimated from related benchmarks
~#14/20~40.8%Estimated
  1. #1Qwen3.8-Omni-Flash92.6%
  2. #13Devstral 244.8%
  3. ~#14Granite 4.2 3BEstimated40.8%
  4. #14Llama 4 Maverick39.7%
estimated from related benchmarks
SciCode
third-party
#16/9755.7%Verified
  1. #1Claude Opus 5.566.9%
  2. #15GPT-5.555.8%
  3. #16GPT-6.1 Sol55.7%
  4. #17GPT-5.6 Terra55.0%
artificialanalysis.ai · 2026-09-29
#1/9766.9%BestVerified
  1. #1Claude Opus 5.566.9%
  2. #2Claude Fable 5.163.1%
artificialanalysis.ai · 2026-09-24
#88/9725.3%Verified
  1. #1Claude Opus 5.566.9%
  2. #87MiniCPM5-2B26.3%
  3. #88Granite 4.2 3B25.3%
  4. #89Ling 3.0 Tiny24.2%
artificialanalysis.ai · 2026-09-24
AA Coding Agent Index
third-party
~#5/15~57.3Estimated
  1. #1Claude Opus 5.566.0
  2. #4Claude Opus 559.7
  3. ~#5GPT-6.1 SolEstimated57.3
  4. #5GPT-6 Sol56.7
estimated from related benchmarks
#1/1466.0Verified
  1. #1Claude Opus 5.566.0
  2. #2Claude Fable 5.162.2
artificialanalysis.ai · 2026-09-24
~last/15below measured range (<41.1)Estimated
  1. #1Claude Opus 5.566.0
  2. #14GPT-6 Luna41.1
  3. ~#15Granite 4.2 3BEstimated<41.1
estimated from related benchmarks
Benchmarks in other categories (4)

Writing & chat benchmarks

Rank among models with a published score, and the leaderboard around each model. ~ italic = no published score, estimated from related benchmarks.

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
IFBench
day 0
~#3/126~82.5%Estimated
  1. #1MiniMax-M382.9%
  2. #2Qwen3.8 2.4T A95B82.8%
  3. ~#3GPT-6.1 SolEstimated82.5%
  4. #3Qwen3.8-Omni-Flash81.5%
estimated from related benchmarks
~#2/126~82.8%Estimated
  1. #1MiniMax-M382.9%
  2. ~#2Claude Opus 5.5Estimated82.8%
  3. #2Qwen3.8 2.4T A95B82.8%
estimated from related benchmarks
—
MMMLU
day 0
~#2/5~90.4%Estimated
  1. #1Gemini 3.1 Pro92.6%
  2. ~#2GPT-6.1 SolEstimated90.4%
  3. #2Gemma 4 31B88.4%
estimated from related benchmarks
~#2/5~92.3%Estimated
  1. #1Gemini 3.1 Pro92.6%
  2. ~#2Claude Opus 5.5Estimated92.3%
  3. #2Gemma 4 31B88.4%
estimated from related benchmarks
—
LMArena Text, Non-English (Elo)
third-party
~#2/54~1499Estimated
  1. #1Claude Opus 5.51501
  2. ~#2GPT-6.1 SolEstimated1499
  3. #2Claude Fable 5.1Few votes1496
estimated from related benchmarks
#1/531501Verified
  1. #1Claude Opus 5.51501
  2. #2Claude Fable 5.1Few votes1496
lmarena.ai · 2026-09-25
—
LMArena Text (Elo)
third-party
~#2/54~1507Estimated
  1. #1Claude Opus 5.51509
  2. ~#2GPT-6.1 SolEstimated1507
  3. #2Claude Fable 5.11501
estimated from related benchmarks
#1/531509Verified
  1. #1Claude Opus 5.51509
  2. #2Claude Fable 5.11501
lmarena.ai · 2026-09-25
—
EQ-Bench Creative Writing v3 (Elo)
third-party
~#7/26~2063Estimated
  1. #1GPT-6 Astra2173
  2. #6GLM-5.32075
  3. ~#7GPT-6.1 SolEstimated2063
  4. #7Claude Opus 5.52050
estimated from related benchmarks
#7/252050Verified
  1. #1GPT-6 Astra2173
  2. #6GLM-5.32075
  3. #7Claude Opus 5.52050
  4. #8Grok 4.72007
eqbench.com · 2026-09-24
—

Vision benchmarks

Rank among models with a published score, and the leaderboard around each model. ~ italic = no published score, estimated from related benchmarks.

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
MMMU
day 0
~#2/3~71.3%Estimated
  1. #1Claude Haiku 4.573.2%
  2. ~#2GPT-6.1 SolEstimated71.3%
  3. #2Llama 4 Maverick69.4%
estimated from related benchmarks
~#2/3~71.3%Estimated
  1. #1Claude Haiku 4.573.2%
  2. ~#2Claude Opus 5.5Estimated71.3%
  3. #2Llama 4 Maverick69.4%
estimated from related benchmarks
—
MMMU-Pro
day 0
#3/7786.0%Verified
  1. #1Claude Opus 5.587.7%
  2. #2GPT-6 Astra86.9%
  3. #3GPT-6.1 Sol86.0%
  4. #4Gemini 3.8 Flash85.6%
artificialanalysis.ai · 2026-09-29
#1/7787.7%BestVerified
  1. #1Claude Opus 5.587.7%
  2. #2GPT-6 Astra86.9%
artificialanalysis.ai · 2026-09-24
—
CharXiv Reasoning
day 0
~#2/9~85.7%Estimated
  1. #1Gemini 3.8 Flash86.2%
  2. ~#2GPT-6.1 SolEstimated85.7%
  3. #2Kimi K384.8%
estimated from related benchmarks
~#2/9~86.1%Estimated
  1. #1Gemini 3.8 Flash86.2%
  2. ~#2Claude Opus 5.5Estimated86.1%
  3. #2Kimi K384.8%
estimated from related benchmarks
—
OmniDocBench v1.5
third-party
~#4/8~90.6%Estimated
  1. #1MiniMax-M391.6%
  2. #2Qwen3.8 27B91.1%
  3. ~#4GPT-6.1 SolEstimated90.6%
  4. #4Gemini 3.1 Pro85.3%
estimated from related benchmarks
~#2/8~91.4%Estimated
  1. #1MiniMax-M391.6%
  2. ~#2Claude Opus 5.5Estimated91.4%
  3. #2Kimi K391.1%
estimated from related benchmarks
—
GDP.pdf (AA)
third-party
~#9/43~21.9%Estimated
  1. #1GPT-6 Astra31.0%
  2. #8Kimi K322.0%
  3. ~#9GPT-6.1 SolEstimated21.9%
  4. #9Claude Opus 521.6%
estimated from related benchmarks
#3/4226.2%Verified
  1. #1GPT-6 Astra31.0%
  2. #3Claude Fable 5.126.2%
  3. #3Claude Opus 5.526.2%
  4. #5GPT-6 Sol24.8%
artificialanalysis.ai · 2026-09-24
—
Chartography
third-party
~#5/23~41.3%Estimated
  1. #1GPT-6 Astra71.0%
  2. #4Claude Fable 5.146.2%
  3. ~#5GPT-6.1 SolEstimated41.3%
  4. #5Gemini 3.8 Flash40.9%
estimated from related benchmarks
#2/2266.3%Verified
  1. #1GPT-6 Astra71.0%
  2. #2Claude Opus 5.566.3%
  3. #3GPT-6 Sol53.6%
surgehq.ai · 2026-09-24
—

Hard reasoning benchmarks

Rank among models with a published score, and the leaderboard around each model. ~ italic = no published score, estimated from related benchmarks.

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
GPQA Diamond
day 0
~#2/174~96.0%Estimated
  1. #1GPT-6 Astra96.1%
  2. ~#2GPT-6.1 SolEstimated96.0%
  3. #2Gemini 3.8 Flash95.3%
estimated from related benchmarks
~#2/174~96.1%Estimated
  1. #1GPT-6 Astra96.1%
  2. ~#2Claude Opus 5.5Estimated96.1%
  3. #2Gemini 3.8 Flash95.3%
estimated from related benchmarks
#125/17355.9%Verified
  1. #1GPT-6 Astra96.1%
  2. #124GLM-4.6V56.6%
  3. #125Granite 4.2 3B55.9%
  4. #126LFM2.5-2.6B55.8%
artificialanalysis.ai · 2026-09-24
Humanity's Last Exam
day 0
#6/18052.9%Verified
  1. #1Claude Opus 5.561.4%
  2. #5GPT-6 Astra54.7%
  3. #6GPT-6.1 Sol52.9%
  4. #7MiMo-V2.6-Pro49.4%
artificialanalysis.ai · 2026-09-29
#1/18061.4%BestVerified
  1. #1Claude Opus 5.561.4%
  2. #2Claude Fable 5.159.1%
artificialanalysis.ai · 2026-09-24
#119/1806.6%Verified
  1. #1Claude Opus 5.561.4%
  2. #118LFM2.5-1.2B-Instruct6.7%
  3. #119Granite 4.2 3B6.6%
  4. #120MiniCPM5-1B6.5%
artificialanalysis.ai · 2026-09-24
AIME (latest)
day 0
~#2/25~96.5%Estimated
  1. #1Inkling97.1%
  2. ~#2GPT-6.1 SolEstimated96.5%
  3. #2Muse Glimmer94.7%
estimated from related benchmarks
~#2/25~96.7%Estimated
  1. #1Inkling97.1%
  2. ~#2Claude Opus 5.5Estimated96.7%
  3. #2Muse Glimmer94.7%
estimated from related benchmarks
~#14/25~44.1%Estimated
  1. #1Inkling97.1%
  2. #13Phi-4-reasoning-plus78.0%
  3. ~#14Granite 4.2 3BEstimated44.1%
  4. #14Mistral Large 338.0%
estimated from related benchmarks
AA Intelligence Index
third-party
#5/7751.8Verified
  1. #1Claude Opus 5.557.6
  2. #4GPT-6 Astra52.7
  3. #5GPT-6.1 Sol51.8
  4. #6Claude Opus 550.8
artificialanalysis.ai · 2026-09-29
#1/7757.6BestVerified
  1. #1Claude Opus 5.557.6
  2. #2Claude Sonnet 5.556.0
artificialanalysis.ai · 2026-09-24
#68/779.1Verified
  1. #1Claude Opus 5.557.6
  2. #67Qwen3 Coder Next9.2
  3. #68Granite 4.2 3B9.1
  4. #69gpt-oss-20b9.0
artificialanalysis.ai · 2026-09-24

Agents benchmarks

Rank among models with a published score, and the leaderboard around each model. ~ italic = no published score, estimated from related benchmarks.

Metric
GPT-6.1 Sol
Claude Opus 5.5
Granite 4.2 3B
τ²-bench (Telecom)
day 0
~#2/116~97.8%Estimated
  1. #1Step 3.7 Flash98.5%
  2. ~#2GPT-6.1 SolEstimated97.8%
  3. #2Gemini 3.1 Pro95.6%
estimated from related benchmarks
~#2/116~98.3%Estimated
  1. #1Step 3.7 Flash98.5%
  2. ~#2Claude Opus 5.5Estimated98.3%
  3. #2Gemini 3.1 Pro95.6%
estimated from related benchmarks
~#55/116~34.4%Estimated
  1. #1Step 3.7 Flash98.5%
  2. #54Sarvam 30B34.5%
  3. ~#55Granite 4.2 3BEstimated34.4%
  4. #55Gemma 4 12B31.9%
estimated from related benchmarks
OSWorld-Verified
day 0
~#3/12~83.4%Estimated
  1. #1Kimi K384.8%
  2. #2Qwen3.8 27B84.3%
  3. ~#3GPT-6.1 SolEstimated83.4%
  4. #3Nex-N2.5-Pro82.2%
estimated from related benchmarks
~#2/12~84.6%Estimated
  1. #1Kimi K384.8%
  2. ~#2Claude Opus 5.5Estimated84.6%
  3. #2Qwen3.8 27B84.3%
estimated from related benchmarks
~#11/12~59.2%Estimated
  1. #1Kimi K384.8%
  2. #10Muse Glimmer65.9%
  3. ~#11Granite 4.2 3BEstimated59.2%
  4. #11Claude Haiku 4.550.7%
estimated from related benchmarks
OSWorld 2.0
day 0
~#3/5~53.8%Estimated
  1. #1GLM-5.3 Flash59.1%
  2. #2Kimi K358.3%
  3. ~#3GPT-6.1 SolEstimated53.8%
  4. #3GPT-5.6 Terra50.2%
estimated from related benchmarks
~#2/5~58.7%Estimated
  1. #1GLM-5.3 Flash59.1%
  2. ~#2Claude Opus 5.5Estimated58.7%
  3. #2Kimi K358.3%
estimated from related benchmarks
~last/5below measured range (<47.5%)Estimated
  1. #1GLM-5.3 Flash59.1%
  2. #4GPT-5.547.5%
  3. ~#5Granite 4.2 3BEstimated<47.5%
estimated from related benchmarks
BrowseComp
day 0
~#5/18~90.5%Estimated
  1. #1Atria Dawn Preview92.5%
  2. #4Claude Opus 590.8%
  3. ~#5GPT-6.1 SolEstimated90.5%
  4. #5Nex-N2.5-Pro89.7%
estimated from related benchmarks
~#2/18~92.3%Estimated
  1. #1Atria Dawn Preview92.5%
  2. ~#2Claude Opus 5.5Estimated92.3%
  3. #2GPT-6 Astra91.5%
estimated from related benchmarks
~#15/18~51.8%Estimated
  1. #1Atria Dawn Preview92.5%
  2. #14Inkling77.1%
  3. ~#15Granite 4.2 3BEstimated51.8%
  4. #15Mistral Medium 3.548.6%
estimated from related benchmarks
GDPval-AA (Elo)
third-party
~#12/87~1612Estimated
  1. #1Claude Opus 5.51846
  2. #11Qwen3.8 Flash-Next1612
  3. ~#12GPT-6.1 SolEstimated1612
  4. #12DeepSeek V4.1 Flash1600
estimated from related benchmarks
#1/861846BestVerified
  1. #1Claude Opus 5.51846
  2. #2Claude Fable 5.11735
artificialanalysis.ai · 2026-09-24
#84/86188Verified
  1. #1Claude Opus 5.51846
  2. #83Ministral 3 8B200
  3. #84Granite 4.2 3B188
  4. #85K2 Think V2117
artificialanalysis.ai · 2026-09-24
AutomationBench-AA
third-party
~#5/80~65.7%Estimated
  1. #1Claude Opus 5.569.5%
  2. #4Grok 4.667.0%
  3. ~#5GPT-6.1 SolEstimated65.7%
  4. #5Grok 4.765.6%
estimated from related benchmarks
#1/7969.5%BestVerified
  1. #1Claude Opus 5.569.5%
  2. #2DeepSeek V4.1 Flash68.9%
artificialanalysis.ai · 2026-09-24
#72/790.4%Verified
  1. #1Claude Opus 5.569.5%
  2. #72Celeris-10.4%
  3. #72Granite 4.2 3B0.4%
  4. #74gpt-oss-120b0.2%
artificialanalysis.ai · 2026-09-24
τ-Bench Banking (AA)
third-party
~#9/90~45.6%Estimated
  1. #1Muse Spark 1.350.5%
  2. #8Kimi K346.0%
  3. ~#9GPT-6.1 SolEstimated45.6%
  4. #9Qwen3.8 Flash-Next45.4%
estimated from related benchmarks
~#3/90~50.0%Estimated
  1. #1Muse Spark 1.350.5%
  2. #2GLM-5.350.3%
  3. ~#3Claude Opus 5.5Estimated50.0%
  4. #3Qwen3.8 2.4T A95B49.1%
estimated from related benchmarks
#77/895.6%Verified
  1. #1Muse Spark 1.350.5%
  2. #75Trinity Large Thinking5.8%
  3. #77Granite 4.2 3B5.6%
  4. #77HyperNova 60B 26055.6%
artificialanalysis.ai · 2026-09-24
Not comparable across labs (2)
OSWorld 2.0 (Sept 2026 release)
day 0
—
#1/381.8%Lab-reported
  1. #1Claude Opus 5.581.8%
  2. #2Claude Fable 5.180.7%
anthropic.com · 2026-09-22
—
OSWorld 2.0 (offline set v2026.08.08)
day 0
#2/471.4%Lab-reported
  1. #1GPT-6 Astra72.6%
  2. #2GPT-6.1 Sol71.4%
  3. #3GPT-6 Sol64.4%
vellum.ai · 2026-09-29
—
—