My Rule for Deciding When to Throw Out AI-Written Code

Ever explained something calmly, in writing, to someone, and they still went and did the exact opposite of what you'd agreed on? That "wait, we already talked about this" moment leaves you unsure whether to laugh or get annoyed. I run into it often when I direct AI to write code.

I work on Work Connect, a product that helps hospitals quickly fill the nursing shifts left open when someone calls in sick. A mistake there isn't just an inconvenience. If a shift goes uncovered, someone has to fix it fast, and the usual options (calling everyone you know, paying an outside agency a premium) are slow and expensive. So there's no room for "we'll clean it up later."

That's exactly the setting where, while building a new feature, the AI I was directing ended up building the exact opposite of what we'd decided to do.

The written requirements come before the code

At Work Connect, before the AI writes a single line, the documents already exist: a product specification, a requirements document per feature with its own acceptance criteria, a technical blueprint, and the work orders that split it into pieces. Only then does the AI turn all of that into code, and I check 3 things: the documents that define what needs to get built, the feature that actually got built, and the tests backing it up.

If code contradicts an approved requirement, I correct the code. I do not rewrite the requirement to justify what the AI built.

The moment I said "this doesn't work"

It happened with an ordinary feature, one that already had its plan fully defined before anyone touched a keyboard. The AI read that plan, wrote the code, and ended up building the exact opposite of what that plan called for.

What came out was a duplicate approach, one we'd already evaluated before and ruled out in writing. The plan didn't just say what to do. In effect, it also said, "we already thought about this, and it's not the way."

When I saw the AI had done it anyway, I didn't hesitate long. I closed that branch of work and went back to the original plan.

Why not just fix what the AI had already built?

Because the code contradicted a decision we had already documented. Patching it would have kept part of that rejected approach in the codebase, where another person or AI session could reuse it without knowing why we had ruled it out.

It wasn't a one-off bug either. What got built went against a decision that was already made, so the implementation needed to return to the original requirement, rather than receive a local patch. A written instruction can be misread again elsewhere; that is another reason to check the result.

When AI code contradicts an approved requirement, I prefer starting over to keeping a patch built on a rejected approach.

The same principle, now inside a team of 6 agents

That episode was about one feature, and I reviewed it myself before it ever reached production.

But Work Connect also has a part that doesn't go through that kind of human, case-by-case review: a team of 6 AI agents that automates a big chunk of the shift-covering process every single day. There, that same rule (never take what the AI did on faith) couldn't depend on me checking every action by hand anymore. It had to become something the system enforced on its own.

Each of those 6 agents handles a piece of work a person used to do manually: they notice when a nurse reports she can't make it in, look for a replacement, screen the candidates who apply, and even make the phone call to confirm someone isn't missing a credential before their shift starts.

The agents start switched off. For the agents that accept delegated rules, a hospital coordinator writes those rules explicitly in plain language.

Before that rule actually takes effect, the panel shows that person exactly what the agent understood, so they can confirm it. And if anything is unclear, the agent doesn't improvise: the shift goes to a manual queue for a person to handle.

This exact tension, how much to trust AI without someone checking all of it by hand, is being argued about in public this year, and I'm not the only one asking it. The reason underneath is simple: an AI can produce work far faster than a person can carefully review it.

For example, curl ended its bug bounty program after being flooded with low-quality AI-generated reports. GitHub has also added controls to disable pull requests or restrict them to collaborators, giving maintainers more control over incoming contributions.

It's not that AI is bad. It's that if nobody sets a limit, speed wins and verification loses.

If you direct AI somewhere a mistake actually matters, you already know what I mean. If you have not run into it, it is still worth deciding how you would respond. The question isn't whether the AI will ever go off plan. It's what you do when it does. Thanks for reading this far. 🙂