Twelve JavaScript frameworks, one task for an AI agent: Angular proves the most expensive, htmx and Astro the leanest

Which web framework lets an AI agent work most cheaply? Nolan Lawson, a developer at security firm Socket, found out by having Claude Code build a simple Pokémon app with the same instructions in twelve frameworks, five times each in a sandboxed environment. He describes the experiment on his blog, and The Register summarised the results. A custom test suite checked that buttons and input fields really worked, while Google Lighthouse measured size and speed.

Hono with htmx was the cheapest, at a median of 2.67 dollars per app, followed by Astro (2.76 dollars, and the fastest build time) and Nuxt (2.82 dollars, with the fewest tokens). The three are within the margin of error. Angular was the most expensive at 4.97 dollars and almost 10 million tokens, and two of its five attempts failed to produce a working app. On loading speed Astro scored best and Next.js and TanStack Start worst, although all apps passed the accessibility and performance tests.

According to The Register, frameworks that do much of their work on the server generally run more efficiently for agents than heavy single-page frameworks in the browser. The publication does note that team familiarity and maintainability weigh at least as heavily in the choice.

Transparency: this edition was compiled by Claude, made by Anthropic.