The Hardest Part of Building With AI Is Knowing When to Stop

Two scripts. That's all this was supposed to be, a quick thing I'd throw away by the end of the week.

The job they automated was genuinely tedious: take a test run and do the whole dance our test-management system demands — create every artifact it expects, wire them together in the right relationships, and report the right results against the right items. Boring, error-prone, exactly the kind of thing you write a quick script for and forget.

A week later, those two scripts were a full command-line tool. And here's the part I'm not proud of: as I write this, my QA team still can't use it. It works perfectly. That's the problem.

From Two Scripts to a Full CLI in One Week

The scripts snowballed into a real tool in about seven days: a setup wizard, importers, reporters, safety checks, a dry-run mode, and rollback logic that could unwind a half-finished import. None of it was junk. Every piece solved a real rough edge I'd hit while using the thing myself.

That's what made it dangerous. Because the original scripts already handled genuine complexity, every new feature felt exactly as justified as the first one. I wasn't gold-plating a trivial tool — I was hardening a useful one. The line between "this is necessary" and "this is me enjoying myself" got very hard to see from the inside.

Why AI Removed the Brake That Used to Stop Me

Over-engineering with AI means building more capability, robustness, or flexibility than the people using the thing actually need — solving problems they don't have yet, at the cost of the ones they have now. It's a specific hazard of AI-assisted development, because for most of my career one thing kept it in check: features were expensive to build.

When a new capability cost me a full day, the price itself forced a question — is this worth a day? Half the time the answer was no, and the feature quietly died in my head before I wrote a line. That friction was doing real work. It was a brake I didn't know I was relying on.

AI took the brake off. According to the 2025 Stack Overflow Developer Survey, 84% of developers now use or plan to use AI tools, up from 76% a year earlier — and when the code appears in minutes instead of a day, nothing pushes back on "one more feature." The cost that used to make me choose is gone.

The data says that missing brake has a price. DORA's 2025 research on AI-assisted software development put it plainly: AI improves throughput, but often at the cost of stability if your foundation isn't solid. You ship faster and break more, unless the discipline underneath — the testing, the review, the boring parts — can keep up. In other words, speed is not the constraint anymore. Restraint is.

The QA Instinct That Turned Into Over-Engineering

Here's the uncomfortable part, and the reason this belongs in a series about building with a QA background: the thing that ran away with me wasn't sloppiness. It was my QA instinct.

Years of testing train one reflex above all others — but what if this breaks? You learn to hunt for the interrupted run, the duplicate artifact, the half-written state, the edge no one thought about. That reflex is what made the tool good. It's also exactly what produced the rollback logic that could unwind a half-finished import — carefully built for a partial failure that had never once happened in practice.

Normally that instinct protects users. Pointed at a tool that no one was using yet, it just protected imaginary ones. I was writing defensive code against ghosts.

The Slow Realization That I Had to Stop and Ship

It wasn't a lightning bolt. It was a feeling that sat with me for two or three days running. I'd finish a session, look at what I'd added, and feel a small wrongness I kept ignoring — the sense that I was moving fast and going nowhere.

On the third day I finally said it out loud: I had built this entire thing and never once put it in front of the people it was for. Everything worked. Nothing was delivered. And those are not the same thing — working software that no one is using isn't done, it's a private hobby with good test coverage.

Momentum Is Not a Roadmap

Momentum feels like progress because it shares progress's texture — commits, features, a growing changelog. But momentum only measures how fast you're moving. It says nothing about whether you're moving toward what anyone needs. A roadmap comes from users. Momentum comes from you.

The discipline I'm relearning is embarrassingly simple:

1. Ship the version that works — even if you can see three more features from where you're standing.
2. Get it into real hands and watch what actually trips people up.
3. Fix the real bugs, not the imagined ones.
4. Let real use pick the next feature — not your own excitement.

This is the flip side of the last thing I wrote about. In why spec-based agentic development starts with QA thinking, the lesson was discipline on the input — writing the specs instead of hoping the AI guesses right. This one is discipline on the output — knowing when to stop and hand it over. Same job, opposite end. Building your own tooling with AI, like the workflow shortcuts I built earlier, makes both kinds of discipline matter more, because the building has never been easier and the stopping has never been harder.

So I'm shipping it this week — unfinished by my standards, which is the point. And here's the quiet proof I got the lesson slightly too late: I ran a few demos for the team, and the feedback was genuinely good. It was good days ago, on a version with fewer features than the one I kept polishing. They didn't need rollback logic for a failure that never happens. They needed the tool.

The hardest engineering decision I make now isn't what to build. AI answers that in minutes. It's when to stop — and I'm clearly still learning it.

What's the thing you can't stop building? If you've caught yourself polishing something nobody's used yet, I'd genuinely like to hear about it — get in touch.