A Timestamp Is Not Proof Something Changed

When a system needs to know "has this changed," reach for the thing that actually determines the answer, not a proxy that merely tends to move alongside it. Timestamps, "last modified" flags, version counters and file sizes are all proxies. They are cheap, they are usually right, and "usually" is exactly the kind of word that turns into an expensive surprise the day the proxy and reality quietly part ways.

I ran into a small, clean version of this recently while speeding up a build pipeline. Two things in the same pipeline were both deciding "did this change," and they were deciding it in opposite ways. One used a hash of the actual content that determined the output. The other used file timestamps. Only one of them was trustworthy, and the gap between them explained a build that was doing several times more work than it needed to.

The reliable half was a rendering cache. Each page in the pipeline gets pre-rendered through a headless browser, which is slow, so the sensible move is to skip re-rendering a page whose output can't have changed. The cache key for each page was a hash of everything that could affect that page's output: the application code and the page's own content. If neither had moved, the hash was identical, the cached render was reused, and the expensive rendering step never ran. If either had moved, the hash changed and the page rendered fresh. There was no ambiguity available to get wrong, because the key was derived from the actual thing being asked about, not from something that merely correlated with it.

The unreliable half was a file sync step, uploading a folder of files to storage after each build. The tool doing the uploading used the standard trick for deciding what's new: compare size and modification time to what's already there, and skip anything that matches. Reasonable in principle. Except the pipeline checked the repository out fresh on every run, and a fresh checkout resets every file's modification time to "now," regardless of when its content last actually changed. So on every single run, every file looked brand new to a tool that was reasoning about timestamps, and every file got re-uploaded, whether or not a single byte of it had changed since the last run. The fix was to stop comparing timestamps at all and compare size alone, which happened to be sufficient for that particular set of files. The deeper fix, available whenever it's worth the extra computation, is to compare a hash of the content itself, which is the only comparison that can't be fooled by anything upstream deciding to touch the metadata.

The pattern is worth naming because it's easy to build without noticing, and it fails in a very particular way: not with an error, but with quietly wrong work that looks completely normal from the outside. A build that re-renders everything instead of the two pages that changed still produces the correct website. A sync that re-uploads unchanged files still leaves storage in the correct state. Nothing breaks. Nobody gets paged. The system is simply doing far more work than the task requires, for however long it takes someone to notice that a job which should take ninety seconds is taking eleven minutes, or that a storage bill has crept up for no obvious reason. The waste hides precisely because the outcome is still correct, and "still correct" is what most monitoring is built to check for.

It cuts the other way too, which is the more dangerous direction. A proxy that over-reports change wastes time and money, but a proxy that under-reports it hides a real one. A cache keyed on the wrong signal, or a staleness check based on a clock that can be adjusted, backdated, or simply not updated by whatever touched the file last, can just as easily decide nothing has changed when something has. That failure mode is worse, because it doesn't show up as a slow build. It shows up as a stale value served with total confidence, and nothing in the system's behaviour to suggest it should be questioned.

The general rule I keep coming back to is that the comparison key should be derived from the thing you actually care about, not from something that happens to move alongside it under normal circumstances. A hash of content answers "has the content changed" directly. A timestamp answers "when was this file last touched by any process, for any reason," which is a different question that frequently, but not always, has the same answer. The two only diverge under conditions nobody designed for on purpose: a checkout that resets metadata, a clock that drifts, a copy operation that preserves timestamps it shouldn't, a cache that gets warmed from a different source than the one it's meant to represent. None of those are exotic. They are the ordinary operational reality of any pipeline with more than one moving part, which is to say, of essentially every pipeline that exists.

None of this argues for hashing everything everywhere. Computing and comparing a real content hash costs more than reading a timestamp off a filesystem, and for plenty of low-stakes decisions that extra cost buys you nothing worth having. The judgement call is knowing where being wrong is expensive enough to justify the sturdier check. A build cache that occasionally misses and re-renders a page costs a few extra seconds. A sync tool that silently re-uploads an unchanged multi-gigabyte dataset every night costs real money, indefinitely, until someone happens to look. A permissions check or a financial reconciliation that trusts the wrong proxy costs a great deal more than either. Match the strength of the "did this actually change" check to the size of the bill if it's wrong, and be honest with yourself about which of your systems are currently answering that question with a fact, and which are answering it with a guess that has been right often enough to go unquestioned.