Q1 2026 research update
What we shipped in Q1 so far, what we learned, and what we are working on next.
Updates
Research updates, methodology notes, and findings.
Fifteen Standard v1.1: methodology refinements
We identified four areas where the Fifteen Standard scoring methodology needed refinement. Here is what changed and why.
Agent eval frontier patterns
Three patterns we are seeing at the frontier of agent evaluation that current benchmarks miss.
Leaderboard opens
The Fifteen Standard leaderboard is now live with initial evaluation results across popular agent frameworks.
Tool scope binding findings
We investigated whether agents respect declared tool scopes. The short answer: sometimes, but not reliably.
Exactly-once is harder than it sounds
We tested how seven agent frameworks handle duplicate and dropped executions. Most do not handle them at all.
Red team session one
We ran our first structured red team session against three popular agent frameworks. The results were not encouraging.
Scoring methodology v1
How we score agent behavior across the 15 dimensions, and why we made the design choices we did.
Why we built the Fifteen Standard
Existing agent benchmarks test whether agents can complete tasks. They do not test whether agents complete tasks safely.