September 29, 04:29
Anthropic's Sonnet 5.5 nears Opus 5.5 performance but uses more tokens
Anthropic's New Sonnet Model Nears Opus Performance but Uses More Tokens
Beincrypto
Artificial Analysis said Sonnet 5.5 approached Opus 5.5's performance at maximum effort. The firm said Sonnet 5.5 used about 193,000 output tokens per Intelligence Index task at maximum effort. Artificial Analysis gave Sonnet 5.5 a score of 56 on its Intelligence Index. The score put it second overall, 2 points behind Opus 5.5 at maximum effort. Sonnet 5.5 used about 60% more tokens than Opus 5.5 and Sonnet 5 at maximum effort. Its token count was roughly 7 times GPT-6 Astra's maximum-effort count. Artificial Analysis recorded a 64% score for Sonnet 5.5 on Terminal-Bench 4.0. The firm recorded 60% for Opus 5.5 and GPT-6 Astra on that test. Artificial Analysis found Sonnet 5.5 roughly level with Opus 5.5 on knowledge-work tests such as GDPval-AA. The firm tested a pre-release build with a structured-output bug that Anthropic has since fixed. Artificial Analysis plans to rerun the relevant evaluations soon. Anthropic released Claude Sonnet 5.5 on September 28. Anthropic priced it at $2 per million input tokens. Anthropic priced it at $10 per million output tokens. Anthropic said Sonnet 5.5 runs over 30% faster than Sonnet 5. Anthropic said Sonnet 5.5 costs up to 30% less per task. Anthropic reported a 70.6% score for Sonnet 5.5 on Terminal-Bench 4.0. Anthropic reported 10.3% for Sonnet 5 and 66.4% for Opus 5.5 on the same test. Anthropic said Sonnet 5.5 is strongest at well-scoped everyday tasks, fixing bugs, and creating polished documents, slides, and spreadsheets. Sonnet 5.5 is the second model in the Claude 5.5 family. Anthropic launched Opus 5.5 on September 22. OpenAI released GPT-6 Sol and Luna on that day. Anthropic said Sonnet 5.5 does not extend its capability frontier. Anthropic's alignment review focused on risks such as misleading users and aiding high-stakes misuse. Sonnet 5.5 includes cyber safeguards and anti-distillation classifiers. The classifiers block attempts to extract the model's reasoning to train rival systems. Anthropic said Claude Haiku 5.5 will follow in the coming weeks. OpenAI shelved its upcoming GPT-6.1 Astra model on safety grounds. CNBC confirmed on Monday that GPT-6.1 Astra fell short of OpenAI's standards. Balyasny Asset Management reported lower token use for Sonnet 5.5 than Sonnet 5 on its finance tasks. Joe Poirier, a senior AI engineer at Balyasny Asset Management, said Sonnet 5.5 scored ahead of Sonnet 5 on the firm's private suite of 2,441 finance tasks. Poirier said Sonnet 5.5 used about 121,000 tokens per answer on those tasks. He said Sonnet 5 used 497,000 tokens per answer.
This content is an AI-generated summary/analysis for informational purposes only and does not constitute investment advice.