The Waste You Only Find Under Pressure

The most expensive waste in a software system is rarely the one anyone budgeted for. It's the quiet duplication that's been running for months, doing no harm anyone can point to, until the day a shared resource turns out to be busy doing something nobody actually asked it to do twice.

I watched a version of this play out recently with a build pipeline. A single commit was quietly triggering two separate build runs — one fired by the code push, another fired by the merge request event a moment later. Both consumed the same limited pool of shared build capacity. Neither run was wrong. Neither failed. The tests passed twice, the artefacts got built twice, and everything reported green, twice, for what should have been one build. Nobody had decided this should happen. It had simply accreted, the way pipeline configuration tends to, through a sequence of individually reasonable choices that nobody looked at together.

It went unnoticed for a long time for a very ordinary reason: nobody was watching for the absence of waste. Engineering attention naturally gravitates to failure. A red build gets investigated within minutes. A build that succeeds twice, silently consuming twice the capacity it needs, doesn't trigger anything, because success isn't a signal anyone monitors for excess. It only became visible on a day when a release genuinely needed to go out quickly and the queue was full. The first assumption was resource contention — too many people building at once, needing more runners. The real answer was smaller and more embarrassing: half the queue was the system building the same thing against itself.

That gap between "it works" and "it costs what it should" is where a huge amount of quiet inefficiency lives, and it isn't unique to CI pipelines. Retried jobs that never needed retrying. Sync processes that re-check state far more often than the underlying data actually changes. Logging pipelines that ship the same field down two different paths because two people solved the same problem eight months apart and neither knew about the other's work. None of these show up as incidents. They show up as a slightly slower system, a slightly higher bill, a slightly longer queue — always slightly, never enough on any given day to justify the hour it would take to trace it properly. So it doesn't get traced, because "it works" is treated as synonymous with "it's fine," and those are not the same claim at all.

What actually surfaced the duplicate build wasn't code review, and it wasn't a scheduled audit. It was scarcity. Someone needed the pipeline to behave well at the exact moment it was quietly behaving twice as expensively as it needed to, and the mismatch between expectation and reality was suddenly worth ten minutes of someone's attention. That's a useful thing to notice about how organisations actually find their own inefficiency: not through diligence, but through pressure. The unglamorous conclusion is that if you wait for a deadline to reveal your waste, you will always find it at the worst possible time — under load, with people waiting, with the fix competing for attention against the actual problem you were trying to solve when you tripped over it.

The fix itself, once found, took minutes: recognise which trigger should own the build, disable the other, done. That's almost always how these turn out. The cost was never in the complexity of the solution. It was in the fact that nobody had a reason to go looking until looking became unavoidable. That asymmetry — trivial fix, expensive discovery — is worth sitting with, because it argues for something most teams don't budget time for: deliberately creating moments of scarcity before a deadline creates them for you. A capacity ceiling set slightly below comfortable, a periodic look at what's actually running versus what anyone remembers configuring, a habit of asking "what would happen if this had to run on half the resources" before you're forced to find out. None of that is thrilling work. It rarely makes it onto a roadmap next to a customer-facing feature. But it's cheaper than discovering your waste live, in front of people who are waiting on you.

There's a commercial dimension to this too, and it's not just the compute bill, although that's real enough once you multiply a doubled build by every commit in a busy repository over a year. The bigger cost is what happens to trust in the system once people notice it behaves unpredictably under pressure. The natural response to a pipeline that sometimes mysteriously runs slow isn't to fix the root cause — it's to work around it. People start manually re-triggering builds "just in case," requesting more runners "to be safe," padding their own estimates because the infrastructure has taught them not to trust its timing. Each of those workarounds is individually sensible and collectively makes the underlying waste worse, because now you're paying for the duplication and for everyone's defensive response to it.

None of this needed to be dramatic to matter. Nobody made a bad decision, nobody was careless, and the system never once told anyone it was wrong. That's precisely what makes this kind of waste worth taking seriously: it doesn't announce itself. It just makes everything a little slower and a little more expensive, indefinitely, for as long as nobody has a reason to ask why. The organisations that find it early aren't the ones with the most sophisticated monitoring. They're the ones who've made a habit of asking, occasionally and on purpose, whether the thing that's working is also the thing they actually meant to build.