The test passes when you run it on its own. It fails when the whole suite runs. Re-running the suite sometimes makes it pass. The usual response is to call it flaky, add a retry, and move on — and that is the wrong response, because the test is not being unreliable. It is reporting, accurately, that something outside it changed underneath it.
A test that behaves differently depending on what ran before it has a dependency it never declared. Finding that dependency is usually a ten-minute job once you know the technique, and the bug it uncovers is often a real one that would have bitten production too.
Below: how to identify the culprit test without reading any code, the three places shared state almost always hides, and the one case where retrying genuinely is the right answer.
Find the pair before you read the code
The fastest route to the cause does not involve understanding either test. It is a bisection: run the failing test with the first half of the suite, then with the second half, and keep halving whichever side still fails. In a few rounds you are down to one other test that, combined with yours, reproduces the failure. That pair is the whole problem, and now the code you have to read is two files instead of two hundred.
Most test runners support running a named subset and, more usefully, running in a fixed or seeded random order. If yours can print the seed it used, a failing run becomes exactly reproducible, which turns the bisection from guesswork into a mechanical procedure.
It is worth doing this before forming a theory. Shared-state bugs are counter-intuitive — the guilty test is often one nobody suspects, doing something innocuous like reading a config value — and a theory formed early tends to send you reading the wrong file first.
Where the shared state actually lives
The first source is module-level state. Anything initialised once when a module is first imported — a cache, a counter, a configured client, a registry of handlers — is shared by every test in the process. One test warms the cache and a later test reads a value it never set. In production this is usually fine, because the process is long-lived and the state is intended; in tests it silently couples files that were meant to be independent.
The second is the database or filesystem. A test that writes a row and does not remove it leaves the next test looking at a table it did not create. This is the one that produces the classic symptom of passing alone and failing together, because alone there is nothing left over. Transaction-per-test with a rollback, or truncation between tests, removes the whole category.
The third is time and randomness. A test that freezes the clock and does not restore it hands the next test a stopped clock. A test that seeds a random generator changes what every subsequent test gets. Both look harmless where they are written and are invisible where they cause damage.
The fix is isolation, not cleanup
The instinct after finding the culprit is to add cleanup to it — delete the row, reset the clock, clear the cache. That works and it is fragile, because it puts the responsibility on the test that caused the mess rather than on the framework, and the next person to write a test that touches shared state will not know to do it.
The durable answer is to make isolation the default: set up fresh state before each test rather than cleaning up after, so a test that forgets to tidy hurts only itself. Fresh-before is strictly better than clean-after, because a test that crashes halfway through never reaches its cleanup, and a suite that depends on cleanup running is one exception away from cascading failures.
Where creating fresh state per test is genuinely too slow — a large schema, an expensive fixture — the compromise is to make the shared thing read-only and enforce it, so tests can lean on it without any of them being able to change it for the others.
When it really is flakiness
There is a real category this does not cover: tests whose failure depends on timing rather than ordering. A test that waits a fixed number of milliseconds for something asynchronous will fail on a loaded machine and pass on an idle one, and no amount of isolation helps. The fix there is to wait for the condition rather than for the clock — poll until the element exists, the queue drains, the file appears — with a generous timeout as the backstop.
The tell that separates the two: an ordering bug fails reproducibly for a given order and passes for others, while a timing bug fails at random within the same order. If you can make it fail on demand by choosing the order, it is not flakiness, and retrying it is hiding a real defect.
One variant deserves its own mention because it looks like neither: the test that passes everywhere except CI. That is usually still an ordering bug, with the runner parallelising across workers so the grouping differs from your machine. Ask the runner to print its worker count and its order, then reproduce that locally rather than debugging through the CI log — a twenty-second feedback loop instead of a six-minute one.
That distinction is worth holding onto, because retries are corrosive when applied to the wrong category. A retry on an ordering bug converts a reliable signal into an occasional one, and the underlying shared-state problem — which can just as easily be a production bug about a cache that is not cleared or a connection that is reused — goes on living.
Why shared global state between tests is the most common root cause of this exact symptom
A test that mutates a shared module-level variable, a shared database record, or any other piece of state not properly isolated and reset between tests can leave that state altered in a way a later test unknowingly depends on or is confused by — run in isolation, the offending test's own side effect simply does not exist yet to interfere with anything, but run as part of a full suite, its leftover state corrupts a completely different, otherwise-correct test that happens to run afterward.
Why test execution order matters more than most test suites assume it should
A test suite that happens to pass reliably only because its tests always run in the same specific order is quietly depending on that specific order, whether or not anyone intended that dependency — deliberately randomizing test execution order on each run, a feature many modern test runners support directly, surfaces exactly this kind of hidden ordering dependency immediately, forcing it to be fixed rather than letting it persist silently until some future change to test ordering finally exposes it.
Why database and external-resource cleanup between tests needs to be genuinely thorough, not just attempted
A test that creates a database record and is supposed to clean it up afterward, but whose cleanup step itself fails silently or only partially succeeds, can leave that data in place to confuse a later, otherwise-unrelated test that happens to query the same table — genuinely thorough test isolation, verified explicitly rather than merely attempted and assumed to work, usually means wrapping each test in its own database transaction that is rolled back completely afterward, guaranteeing a clean state for the next test regardless of what the current one actually did.
Why this specific failure mode is disproportionately expensive to debug precisely because it is intermittent
A test that fails exactly the same way every single time is straightforward to debug, while a test that only fails intermittently, and specifically only when run alongside certain other tests in a certain order, is one of the more time-consuming categories of bug to track down, since reproducing the failure reliably enough to actually investigate it often requires first identifying the exact specific combination and order of tests that triggers it in the first place.
Detecting the problem before it becomes a mystery
The cheapest time to catch this class of bug is long before someone is staring at a baffling failure in CI, and the fix is almost embarrassingly simple: run the test suite with randomized test order as a matter of routine, not as a special diagnostic step reached for only after something has already gone wrong. Most modern test runners support a seed-based randomization flag, and turning it on by default in CI means any hidden ordering dependency surfaces as a flaky-looking failure early, while the change that introduced it is still fresh and easy to connect to the failure.
The much worse alternative is a team that runs tests in a fixed, alphabetical or file-discovery order for years, during which a subtle ordering dependency can be introduced, silently rely on it working, and only be discovered when an unrelated refactor happens to change file discovery order or someone adds parallel test execution — at which point dozens of tests can start failing at once with no obvious common cause, because the common cause is an assumption nobody wrote down or even noticed they were making.
Database state as the most common hidden dependency
Among the many forms of shared state that cause this, a database left in whatever condition the previous test left it in is the most common offender in real codebases. One test creates a user record and never deletes it; a later test that counts users or searches by an assumed-unique email collides with that leftover row, and the failure only appears when both tests happen to run in the same process against the same database in that particular order.
The fix that scales is wrapping each test in a database transaction that gets rolled back at the end, so no test's data can ever leak into the next one's view of the world regardless of execution order. Teams that adopt this pattern early rarely encounter order-dependent test failures at all, because the shared resource that would have carried the dependency simply resets itself every time.
CI parallelization as an accidental stress test
Teams that introduce parallel test execution in CI purely for speed often get an unplanned side benefit: running test files across multiple workers effectively randomizes which tests share a process and in what order, which tends to surface exactly this class of hidden-dependency bug faster than a slower, sequential CI run ever would. It is worth treating any new failure that appears right after enabling parallel test runs as a strong candidate for a pre-existing ordering bug that parallelization simply exposed, rather than assuming the parallelization itself introduced a new defect.