Why My Tests Passed While My Blog Was Broken in Production

Ever checked something carefully, seen green lights everywhere, and still ended up with something broken? That happened to me with my own website. I wrote the code, ran every automated check I have set up, and all of them passed. The problem is that none of those checks were looking at the same thing a real visitor was about to see.

This site is my portfolio, and I build it by directing AI through a process with written rules and automated checks before anything counts as done. The checks cover page appearance, accessibility and whether the code compiles without errors. The blog section passed every one of them.

And yet every post I published broke completely the moment a real person opened it. The page showed a generic error message. Not a single word of the article showed up.

What I took away

• An automated check only tests what it actually runs. If it runs in an environment more permissive than the real one, it can say everything is fine when in production it isn't.
• The easiest fix isn't always the right one, especially when it weakens something as important as security.
• Fixing a visible bug can uncover an invisible one that was living right next to it.

How my blog ended up showing a generic error

The blog's content is written in a format that mixes regular text with small pieces of code. It's called MDX. Before you can read it, something has to turn that format into what a browser understands.

My site has a security policy that blocks evaluating text as executable code inside the browser, a technique used by eval. Blocking it limits what an injected script could do. It turned out the piece that converted the blog's format into readable text used exactly that technique, inside every visitor's own browser.

Why my tests never caught it

Every one of my tests ran against a development version of the site, and that version does allow the blocked technique, because it needs it for something else: refreshing the page while I'm coding, without a full reload. Nobody decided that on purpose for the blog. It was an old assumption that never got checked again: that only that other thing needed the permission.

So my accessibility tests passed. My visual tests passed. The check that the page loads without errors passed. All of them ran against the version that has the permission, never against the real version a visitor sees.

Why not just allow that technique in production?

Because allowing it would weaken a protection across the site to solve one problem in the blog. I wanted to keep that restriction and change how the content was compiled, so the browser would not need to evaluate code from text.

A fix that weakens the whole site's security to solve one section's problem isn't a fix. It's debt you'll pay somewhere else.

Fixing this uncovered a second, hidden bug

The real solution was to move that conversion to the server, where compilation no longer depends on permissions in the visitor's browser. The blog's text reaches the visitor already converted, ready to read.

That change fixed the main problem, but it uncovered a second one. One of the blog's code blocks had a button to copy it with a single click. That button lived in the part of the site that runs in the browser. Once I moved the conversion to the server, that piece stopped connecting correctly, and the button disappeared without showing an error.

I found it by comparing the new version against the old one, line by line, not because anything warned me. I now have 2 tests dedicated exactly to this: one that checks that no part of the browser-side code tries to do this conversion, and another that opens a real post and confirms the copy button is still there.

If you've ever seen everything green and still had something break in production, you already know that feeling. For me, the useful lesson was to test the environment visitors actually use, and check what the fix might have changed alongside it. Thanks for reading this far. 🙂