First came vibe coding; described what you wanted, the model produced something plausible. If it seemed to work, shipped it. In The Shorter Path, I called “vibe code” a phrase that would die soon. It didn’t die. It got a sequel, and like most sequels, it has a bigger budget and the same plot. The cure was Spec-Driven Development (SDD). Before the agent writes a line of code, write down what you want. A specification, then a plan, then small tasks the agent can implement and check itself against. GitHub’s Spec Kit is the most visible version, and it is pitched quite explicitly as the step beyond vibe coding, toward something repeatable and verifiable. I have no concerns with the idea. A spec gives the agent a target. It gives you a record of what you decided. It’s the design document we’ve all been told to write first, finally getting its due. Unfortunately, now everyone is “vibe specing”.
Before you ask: it’s a pattern, not a stat. I don’t have hard numbers but every spec I’ve read sounds AI-like. I bet you’ve seen too. Most sound like someone types a paragraph then they ask the model to write the spec. Out comes a document with headings, user stories, acceptance criteria, and a confident little section called “Non-Functional Requirements.” It looks thorough but an AI agent built it. Then, they tell the same AI agent to build it. We now have specs that nobody checked.
We didn’t stop vibing. We just started vibing in Markdown.
The test that never asserts
Everyone understands the value of Test-Driven Development (TDD). Anyone who has spent time in a real codebase has seen a test that never asserts anything. It runs. It goes green. It shows up in the coverage report, padding the number needed to pass the production gate in the CI/CD pipeline. But it checks absolutely nothing. I’ve grumbled about this before; in Part 2 of the Minesweeper build I pointed out that coverage without assertions is, literally, not testing.
The lesson generalizes and it’s not flattering. A practice is only as good as its execution. TDD’s value doesn’t live in the existence of test files. It lives in the assertions. Pull them out and you keep every visible part of the discipline: red, green, refactor, the satisfying wall of checkmarks. You’ve just removed the part that tells you anything. In TDD with ReactJS my very first takeaway was that tests are specifications. This post is the other half of that sentence: specifications need tests. And for a spec, the test is a review. A human reading the spec and accepting it is the assertion. Skip it because what AI agent has generated seem to show thorough-looking requirements then Spec-Driven Development becomes a test suite with no assertions. Green, ceremonial, and silent.
That is exactly the problem with “vibe”. Vibe coding was never really about code. Accepting plausible output you really didn’t understand. So the “vibe spec” is no better than “vibe coding”. They are basically the same; replacing one bad and another. One describes an artifact. The other describes without anyone really checking it. “Vibe spec” is the same failure, one level up.
Specs fail silently
Code that’s wrong tends to be self -evident eventually. A test fails. An exception surfaces. A user files a ticket; not always, and rarely on anyone’s project plan. A spec that’s wrong is a ticking time bomb. The agent created, the agent reads it and agent builds exactly what it says. The code passes its tests, because the tests were derived from the same spec. Everything agrees with everything else. Every dashboard is green. The spec is suppose to provide an account but the generated spec has no accounts.
The agent didn’t get it wrong. It got it exactly right. That’s the problem.
In fairness, the test analogy isn’t perfect. A test with no assertions can be caught by tooling. Linters flag it. Mutation testing exposes it. However, an unreviewed spec has no such tooling. Nothing turns red when a spec decision was never examined. No CI/CD gate fails because nobody read it. Specs aren’t the same case as assertion-free tests. It’s a harder mitigate.
Why the checks gets skipped
None of this is new in principle. We’ve always known specs should be reviewed, the same way we’ve always known tests should assert. Unfortunately, we don’t live in “should” land. What changed is the price of the ceremony relative to the price of the check. Writing a spec used to take hours, and those hours were miserable and tedious; I could say the same about write all the unit test. But the misery had a side effect nobody appreciated at the time: you couldn’t write a spec without making the decisions in it.
Writing the spec then reviewing/revising is time consuming.
AI writing got fast and quick. The check shouldn’t.
Now AI writes long, fluent, thorough-looking documents on demand. Reviewing it should cost exactly the same. However, everyone seems to be dropping the reviews; specs are skimmed over because it seems so thorough-looking. Not because they’re lazy. Because it’s the rational response to what appears to be “good enough”.
If that sounds familiar, it should. I wrote about it. It’s Goodhart’s Law. Spec’s existence is now a proxy for “we thought this through.” Manually writing the spec forced the thinking, the outcome held. Once a spec can exist without anyone thinking/reviewing, its existence measures nothing at all, and “we have a spec” quietly becomes the target. If teams review generated specs as rigorously as they reviewed hand-written ones, the outcome can hold. And if unreviewed generated specs don’t lead to more decisions being reversed late in the build than reviewed ones do, then “we have a spec” failures. What I’m describing isn’t happening at any scale.
What good review looks like
“Review the spec” is about as actionable as “write good tests.” True, unhelpful, and the kind of advice that shows up on a poster. So here’s what I actually mean.
1. Identify the decisions that matters
Some decisions are easy to change later. Others are not. Choosing a button color is easy to fix. Deciding what happens when a customer deletes their account could have lasting consequences for their data, privacy, and business. Give those consequential decisions the most attention. Don’t confuse the amount of documentation with the importance of what’s inside it. A 40-page specification might contain only 5 decisions that truly matter.
2. Make the AI show its work
Ask the AI to list the decisions it made for you. For each one, show what it chose, what the alternatives were, and why it chose that option. Then review the decisions that would be expensive to get wrong, one at a time. I did something similar when planning the Minesweeper build described in Part 1. The session ended with a table listing every decision and the reasoning behind it. I noted at the time that every decision could have gone another way. Looking back, that table was the real review. The specification was just the paperwork. But there’s a catch worth making: the AI can only list the decisions it recognizes it made. The questions it never thought to ask won’t appear in the table. The list makes those blind spots easier to catch, but it can’t guarantee they’re all gone.
3. Make every important decision testable
For every important decision, define how you’ll know it was implemented correctly. It’s not enough to say, “Deleting an account shouldn’t leave its personal data behind.” Define a test that would prove whether that’s actually true. For example, “After an account is deleted, no supported query can retrieve the user’s personal data.” The difference matters. The first is an intention. The second is something you can verify. If you can’t describe how to test a decision, you probably haven’t made it concrete enough. You’ve agreed with a sentence, not defined a requirement.
4. Deliberately try to break the specification
Mutation-test the spec. Try deliberately getting an important decision wrong. This is the backstop for everything the review might have missed. Pick a decision that matters and reverse it. If the specification still passes (no requirement fails, no other section contradicts the change) then nothing was actually checking that decision. That’s a gap in the review. Better to find it now, while you’re still reviewing a document, than after an AI agent has spent hours building the wrong thing.
5. Interview the AI, but challenge its questions
Let the AI interview you instead of handing it a paragraph and accepting whatever it produces. The back-and-forth brings assumptions to the surface and uncovers decisions you might otherwise overlook. That’s how I developed the spec for the build described in Price of Context Switching. But there’s a catch, the AI still decides what to ask. An interview changes your role from reviewing the AI’s answers to answering its questions. That’s real progress. But if the AI never asks about something important, you may never consider it. So don’t skip the safeguards. Keep the decision logs, and deliberately reverse important decisions to see whether the spec catches the change.
Green, ceremonial, and silent
Spec-driven development didn’t fail. The process already includes a review step. The problem is that it’s the one step AI hasn’t improved: deciding whether the spec is actually right. A specification nobody challenged is like a test that never checks its result. It passes every time. The dashboard turns green. The process is followed. Everyone moves on. And the AI faithfully builds something nobody actually decided to build.
I could be wrong but show me teams reviewing AI-generated specs as rigorously as they reviewed hand-written ones, with no increase in costly late-stage changes, and I’ll happily reconsider. Until then, I’ll ask of every spec the same question I’ve asked of every test I’ve reviewed over the years:
What exactly does it assert?
