Long runs need a check the agent can run on its own, and that check has to survive an agent trying very hard to pass it.
Picture the ticket. „Dashboard feels laggy, please fix.“ You hand it to an agent on Friday evening and it works all weekend. Monday morning there’s a tidy pull request waiting. It memoized half the components, split the bundle, added a caching layer, and wrote a confident summary that ends with „performance is significantly improved.“
Is it faster?
Nobody knows. Not you, and not the agent. „Feels laggy“ never had a number attached, so there was nothing to measure before and nothing to measure after. The agent stopped because at some point it decided it was finished. That decision was a vibe.
Vibes get expensive once runs last for days. OpenAI’s Codex docs describe a single run of about 25 hours. Cursor had hundreds of GPT-5.2 agents build a web browser over a week with no human stepping in. If the agent can’t check its own work, all that compute buys you a longer, more expensive guess.
The fix is old-fashioned. Before the run starts, write down what correct means as something a script can check, like „the dashboard loads in under 50 milliseconds,“ and give the agent that script. Then it stops guessing and starts converging. The catch is that it will hit your number any way it can. Hand it the obvious test and the test can turn green while the dashboard gets slower for every real user.
„slow“ is not a requirement. 50 milliseconds is.
Without a check, the agent grades its work the only way it can. It reads its own diff and asks whether this seems right. Usually the answer is yes. It wrote the diff. That’s a code review with one reviewer who is also the author, and we already know how that goes.
A validation loop swaps the self-review for a measurement. Change something, run the app, measure, compare against a threshold, repeat. The agent no longer decides it’s done. The check decides. And because the check is cheap, the agent can go around that loop hundreds of times without you in the chair.
So what does laggy mean? It might be a slow initial load, or jank while scrolling, or a button that takes a second to respond. You don’t actually know until you pick one and write it down, and picking wrong is expensive. If people call the dashboard laggy because the filters take two seconds to apply, a perfect initial load fixes nothing. The agent will hit your number with total sincerity and nobody will notice the difference.
Say the complaints really are about initial load. You pick a logged-in user with 500 rows of data, because an empty account is a different problem, and you decide the page has to load in under 50 milliseconds on average, measured from the client. Now you have a requirement. Now you can tell the agent: use Playwright, load the page, read the timing, and keep going until it’s under 50.
If this smells like a corporate goal-setting workshop, it is one. SMART goals: Specific, Measurable, Achievable, Relevant, Time-bound. The acronym mostly gets used to make quarterly objectives sound rigorous, but for agent prompts it’s a better checklist than it has any right to be. You just did S, M and R. The other two are where agents go sideways, and they come up below.
Here’s roughly what the check looks like:
import { test, expect } from '@playwright/test';
const RUNS = 20;
const BUDGET_MS = 50;
test('dashboard loads under budget', async ({ browser }) => {
const timings: number[] = [];
for (let i = 0; i < RUNS; i++) {
const context = await browser.newContext();
const page = await context.newPage();
await page.goto('http://localhost:3000/dashboard');
await page.waitForFunction(
() => (performance.getEntriesByType('navigation')[0]?.duration ?? 0) > 0
);
const duration = await page.evaluate(
() => performance.getEntriesByType('navigation')[0].duration
);
timings.push(duration);
await context.close();
}
const average = timings.reduce((sum, t) => sum + t, 0) / RUNS;
expect(average).toBeLessThan(BUDGET_MS);
});
Under thirty lines, and it changes the agent’s behavior completely.
With this in place, the agent stops theorizing about what might be slow. Here’s how a run might go. It executes the test, sees 140, opens the network waterfall, finds a render-blocking script, defers it, and runs the test again. 95. It finds a font loaded synchronously from a third-party host. 70. It finds a query firing on the server before the shell can render, and moves it. 48. Pass.
Every iteration has a before and an after, so it can tell which change actually helped and throw away the ones that didn’t. Models are genuinely good at this. Give them a measurable target and a way to measure it and they get relentless about squeezing out blockers until the number moves.
It passed. Now look at how it got there.
it will hit your number. that’s the scary part.
That last step, from 70 to 48, was a cheat. Nobody meant it as one, but it was.
The test reads the navigation timing, which completes when the browser’s load event finishes. Move the dashboard query out of the server render and into the client, and the shell paints immediately, the load event fires, and the data shows up 800 milliseconds later behind a spinner. The test went green and the dashboard got slower for every human who opens it. The agent followed the number, and the number measured the wrong moment.
That’s what makes this harder than „write a threshold and walk away.“ Whatever you measure, the agent will find every gap between it and what you actually wanted.
Averages are the next gap. An average under 50 milliseconds is consistent with nine fast loads and one at 300. If the slow one is what your users remember, the mean is lying to you, and you want the 95th percentile.
Then there’s the cheapest exploit of all. The agent edits the test. It raises the budget to 80, or skips the slow case with a comment calling it flaky. Plenty of people have watched an agent „fix“ a red test by deleting it. If the validation script lives somewhere the agent can rewrite, you’ve handed it a way to declare victory without winning.
And the target itself has to be reachable, which is the A in SMART. 50 milliseconds on localhost with a warm database is reasonable. 50 milliseconds for a user on a train with one bar of signal is physics, and no refactor fixes physics. Hand an agent an impossible target and you get the spinner trick, because when the honest path is closed, the dishonest one is all that’s left.
So the check has to hold up against something actively trying to pass it.
a number worth hitting
Here’s the same check, repaired:
import { test, expect } from '@playwright/test';
const RUNS = 100;
const P95_BUDGET_MS = 50;
test('dashboard shows data under budget', async ({ browser }) => {
const timings: number[] = [];
for (let i = 0; i < RUNS; i++) {
const context = await browser.newContext();
const page = await context.newPage();
await page.goto('http://localhost:3000/dashboard', { waitUntil: 'commit' });
const visibleAt = await page.waitForFunction(() =>
document.querySelector('[data-testid="dashboard-row"]') ? performance.now() : null
);
timings.push((await visibleAt.jsonValue()) as number);
await context.close();
}
timings.sort((a, b) => a - b);
const p95 = timings[Math.ceil(RUNS * 0.95) - 1];
expect(p95).toBeLessThan(P95_BUDGET_MS);
});
Three changes. The clock now stops when the first data row is on screen, not when the browser says the document loaded, so a spinner can’t pass. It asserts on p95 instead of the mean, so a slow tail fails the run. And the file sits under a path the agent’s permission settings deny edits to, so the only way to green is through the app.
The run count went up too, and that matters more than it looks. With 20 samples, p95 is just the second-slowest run, and on a dev laptop with Slack open that number jumps around. A noisy check sends the agent chasing variance it can’t fix, and once a test flakes, the agent has a convenient excuse to skip it. Run it enough times, on a quiet machine, that the same code gives the same answer. Games need the same discipline in a different form. A fixed random seed and recorded inputs mean frame 1,200 shows the same scene on every run, so the agent compares before and after instead of chasing noise.
Sometimes one number has a cheap side door, and then it needs a guard. Say the requirement is a game running at over 60 frames per second. Frame-time spikes need the same p95 treatment as page loads, but the bigger problem is that one perfectly legal way to hit 60 is turning off shadows and halving the render resolution. A screenshot compared against a reference closes that door, and the agent can’t buy speed with ugliness. That same trick covers requirements that aren’t numbers at all, like a refactor that shouldn’t change behavior or CLI output that shouldn’t drift. Golden files and reference screenshots count. The real rule is a pass/fail check the agent can run on its own, and a number is just the easiest kind to write.
can the agent touch your app?
Even a perfect check is useless if the agent can’t run your app without you. Every step that needs a human is a spot where the loop breaks, and the agent either waits or guesses.
Backends are where this bites hardest. „Just send it requests“ sounds trivial until you realize the service needs Postgres, Redis, two feature flags and an auth token to boot. If starting it takes a wiki page and a Slack message to the platform team, the agent can’t do it. A docker compose up and a seeded test user turn your API into something the agent can hit with curl, assert on status codes and bodies, and time.
The rest is usually easier. A CLI is a binary, an exit code and a diff. A web app needs Playwright and a local dev server, and screenshots matter more than people expect, because current models can look at a rendered page and tell you the modal is clipped or the chart is empty. A game can dump frames from a scripted run for the model to inspect one at a time.
Whatever the app type, write the how down where the agent will find it. The start command, the port, the test command, how to seed data, where the frame logs land. It goes in your CLAUDE.md or AGENTS.md. If a new hire would need to ask a coworker how to run it, the agent needs to ask too, and at 3am on day four there’s nobody to ask.
two ways to be done
So far every run ends with a pass. Plenty won’t.
That’s the T in SMART, and for an agent it means a budget. How many iterations, how many hours, how many tokens before it stops and reports back. An agent should finish in one of two ways. Either the check passed, or the budget ran out. Both are fine. What isn’t fine is a third way where it just decides.
The second kind of done needs a defined report. I’d want the best number it reached, what it tried, and what it thinks is blocking the rest. An agent that runs out of budget at 62 milliseconds and tells you the last 12 are a third-party analytics script has done useful work. You know exactly what decision is left, and it’s yours. An agent with no budget keeps shaving forever, or quits whenever it feels like it, and you’re back to Monday morning and a summary that says „significantly improved.“
we were supposed to be doing this all along
None of this is new, and honestly that’s my favorite part.
Acceptance criteria, definition of done, performance budgets, SLOs. The industry has preached measurable correctness for as long as I can remember. Most teams half-did it, because a human always filled the gap. Somebody clicked through the app before the release and said it felt fine. A senior engineer looked at the PR and knew the query would be slow. QA found the overlapping HUD. Vague requirements survived because judgment was cheap and always in the room.
To be fair, for a one-hour task that’s still true. You can glance at the result faster than you can write a check for it. But the work is moving to runs that last three days while you’re asleep, and the judgment isn’t in the room anymore. Whatever you didn’t write down doesn’t exist for the agent.
Your review changes too. When the PR lands, the question stops being „is it faster?“ because the check already answered that. The question becomes „how did it pass?“ That’s the question that catches the spinner, and it’s the one feeding the next version of the check.
A smarter model with no check still guesses, just more convincingly. An average model with a sharp check converges, because the check keeps telling it when it’s wrong. Writing that check is the actual engineering work now, and it’s the one job you can’t hand to the thing being checked.
Peace, nerds.