Outside evaluator METR finds Anthropic improving its models at about double the earlier rate, though much of the evidence is redacted

The US non-profit METR on Thursday published an evaluation of how quickly Anthropic is improving its AI models. From 12 August to 29 October, METR staff had access to internal benchmark figures, data on compute and headcount, employee surveys and around fifteen interviews. Their conclusion: on clearly defined tasks, progress is now probably running at roughly twice the pace seen between March 2024 and October 2025, within a range of 1.3 to 3 times.

According to the report, an estimated 80 percent of that acceleration is explained by the company's own use of AI in research, such as help with coding and analysing experiments. METR says more training compute is not the cause. It found no convincing bottlenecks for the next twelve months, but says its understanding of them changed several times. Fully autonomous, decisive contributions by AI to the most important training work have not yet happened, although researchers reported early signs.

Anthropic had the right to cut the report of around one hundred pages unilaterally and removed most of what METR considered most informative, including estimates of causes, bottlenecks and forecasts. Co-founder Sam McCandlish told the AP news agency that he largely agrees with the conclusions and defended the redactions as commercially sensitive, while acknowledging that they make outside verification harder. CryptoSlate points out that success on easily measured tasks does not automatically translate into usefulness in practice.

On the LessWrong forum, researchers were divided. Some see an important sign that powerful AI is arriving sooner, while others argue the figures point to a one-off kink in the trend rather than self-reinforcing acceleration, noting that Claude itself helped process much of the data. Transparency: this edition was compiled by Claude, made by Anthropic.