Can't Reproduce That Bug? Stop Guessing and Start Isolating

About half of all software development budgets go to testing, yet software flaws still cost the U.S. economy $59.5 billion annually (NIST, 2010). That's a staggering number, and it points to a dirty secret: most of that waste isn't from bugs slipping through—it's from bugs we can't reproduce. When a bug won't show up on demand, it sits in limbo, eating time and morale. The fix isn't more testing. It's better reproduction.

Why Reproduction Is the Real Bottleneck

Every defect moves through a lifecycle: New, Assigned, In Progress, Fixed, Pending Retest, Retest, Verified, Closed (Atlassian). But there's a graveyard state that kills momentum: Non-Reproducible. That's where bugs go to die—or worse, to resurface in production.

The problem is that most reports are incomplete. A complete defect report includes reproduction steps, expected vs. actual behavior, severity, priority, environment details, and visual proof (Atlassian). Yet how many of us actually get that? Usually, we get a one-liner: "It crashes when I click the button." Then we spend hours trying to guess which button, which browser, which data.

Here's my point: reproduction isn't a step in the process—it's the foundation. If you can't reproduce it, you can't fix it. So stop treating reproduction as a passive outcome and start treating it as an active discipline.

The Isolation Principle: Control the Variables

The single most effective technique I've found is variable isolation. You change one thing at a time until the bug appears or disappears.

Start with the environment. Was it a specific browser version? A particular operating system? A mobile device? The NVD enriches every CVE with Common Platform Enumeration (CPE) applicability statements (NVD), which means even security folks track which platforms are affected. You should do the same for your bugs.

Next, look at the data. If the bug is data-dependent, you need to know the exact input. The NIST's ACTS research shows that virtually all software failures are caused by six or fewer parameter interactions (NIST, 2010). That means you don't need to test every combination—you need to find the critical few. Apply that to reproduction: don't ask the user to "describe what they did." Instead, ask them to go back and deliberately change one variable at a time.

For example, a user reports a crash on checkout. You ask: "Does it happen with a promo code? Without? Does it happen in Chrome but not Firefox? Does it happen with a logged-in user?" Each answer narrows the space. You're not guessing; you're isolating.

Make Your Bug Tracker Work for You

Your tracker isn't just a database—it's a reproduction tool. Use custom fields to capture environment details, steps, and expected vs. actual behavior. Bugzilla exists precisely because teams outgrow built-in tracking in source-code management systems, typically for workflow management, bug-visibility control, or custom fields (Bugzilla). Custom fields are your friend.

But fields alone don't help if the workflow is chaotic. A simple, effective workflow is Open to In Progress to In Review to Done to Closed, optionally adding a Triage step (Atlassian). Triage is where you can separate reproducible bugs from vague ones. If a bug lacks reproduction steps, don't assign it—send it back to the reporter with a specific request.

And here's a radical idea: if you can't reproduce a bug in 30 minutes, stop trying. Log it as Non-Reproducible, but with a detailed report of what you tried. That way, if it happens again, you have a starting point. The worst thing you can do is spend days on a bug that won't show up, while other bugs pile up.

What to Do When You Can't Reproduce

Sometimes, despite your best efforts, the bug is a ghost. That's when you need to think like a detective, not a developer.

First, check the severity. If it's a Fatal/Blocker that crashes the system, you must pull out all stops. But if it's Minor, you might have to accept it for now. Priority measures urgency, severity measures impact (Atlassian). A Minor bug that blocks a release is a priority, but a Blocker that only appears on an obsolete browser might be lower priority.

Second, consider the environment. The Mars Global Surveyor mission ended after 9 years because a software update was stored at the wrong memory address (NASA). That's a one-in-a-million environmental glitch. If your bug is environment-specific, you need to replicate that environment exactly. Use virtual machines, Docker containers, or old hardware if necessary.

Third, involve the user. Ask them to record a video or provide a screenshot. Visual proof is part of a complete report (Atlassian). Sometimes, just seeing the user's screen reveals the problem—maybe they're using a browser you didn't consider, or they have a strange resolution.

Finally, don't be afraid to close it. Mark it as Non-Reproducible, but add a note: "If this happens again, please capture the console logs." That's not failure—it's pragmatic resource allocation.

Bottom line

The best move you can make is to adopt variable isolation for every bug report. Force yourself to change one variable at a time, and you'll turn most Non-Reproducible bugs into reproducible ones. The $59.5 billion problem won't vanish overnight, but you'll cut your share of the waste.

Sources

• Atlassian - https://www.atlassian.com/software/jira/guides/bug-tracking/bug-life-cycle
• NIST - https://www.nist.gov/news-events/news/2010/11/updated-nist-software-uses-combination-testing-catch-bugs-fast-and-easy
• Bugzilla - https://www.bugzilla.org/about/
• NASA - https://ntrs.nasa.gov/api/citations/20120001915/downloads/20120001915.pdf
• NVD - https://nvd.nist.gov/general