Why Your Bug Reports Need a Reproduction Recipe, Not Just Steps

What's the one question that decides whether a bug gets fixed?

It's not "What's the severity?" or "Who should own this?" — it's "Can you reproduce it?" A bug that can't be reproduced is a ghost. It lingers in the tracker, gets tagged "Non-Reproducible," and eventually gets closed. The harsh truth is that most bug reports fail not because the bug doesn't exist, but because the reproduction steps are incomplete or ambiguous. We've all been there: a QA analyst files a bug with "Steps to reproduce," but the developer tries it and nothing happens. The report bounces back. The cycle repeats. The bug festers.

So, what's the real fix? It's not about writing longer steps. It's about writing a recipe — a precise, repeatable procedure that isolates the exact conditions under which the bug appears. And there's a body of research that tells us exactly why this matters: NIST found that 70 to 95 percent of software failures are triggered by only two interacting variables, and practically 100 percent are triggered by no more than six (NIST). That's a staggering insight. It means that most bugs aren't caused by a single input; they're caused by a combination of conditions. So if your reproduction steps don't capture those combinations, you're setting the developer up to fail.

In this article, we're going to argue that the heart of effective bug tracking is a rigorous reproduction process, and we'll give you a concrete approach to writing reproductions that actually work.

What does a good reproduction actually look like?

We often think of reproduction steps as a simple list: "1. Go to page X. 2. Click button Y. 3. See error." But that's a script, not a recipe. A recipe specifies ingredients, quantities, and conditions. A reproduction that works must do the same. It needs to specify the environment (browser, OS, screen size, network state), the data (specific user accounts, database states), and the sequence of actions with exact timing or preconditions.

Consider a bug that only appears when a user is on a slow connection and clicks a button twice quickly. If your steps say "Click the button," the developer might not reproduce it because they're on a fast connection and click once. But if you say "Throttle network to 3G, log in as test user, and double-click the submit button within 500ms," you've provided a recipe. That's the difference.

In practice, we've found that the best reproductions are written with the developer's mindset: they are minimal, deterministic, and include the expected vs. actual behavior. The Atlassian bug life cycle guidance emphasizes that a complete defect report includes reproduction steps, expected vs. actual behavior, severity and priority, environment details, and visual proof (Atlassian). That's a good checklist, but we'd argue the reproduction steps are the linchpin. Without them, the other fields are just decoration.

Why do so many reproductions fail?

The biggest reason is that we, as reporters, assume too much. We assume the developer has the same data, the same permissions, the same network conditions. We assume that our "step 2" is self-evident. But software is a system of interacting variables. NIST's research on combination testing is a wake-up call: it shows that most failures involve two to six interacting parameters (NIST). So if your reproduction doesn't explicitly control those parameters, you're leaving it to chance.

Another common failure is the heisenbug — the bug that disappears when you try to observe it. This often happens because the act of debugging changes the timing or state. For example, a race condition might only occur when the system is under load, but when you open the debugger, the timing changes and the bug vanishes. That's a reproduction problem, not a bug problem.

We've also seen a cultural issue: reporters sometimes feel that if they've described the bug clearly, that's enough. They don't want to spend time crafting a reproduction. But that's a false economy. A bug that can't be reproduced will cost far more time in back-and-forth than it would have taken to write a good reproduction in the first place. The 2002 NIST study estimated that inadequate software testing costs the U.S. economy $59.5 billion annually (NIST). While that's a macro figure, it underscores the cost of poor bug-handling processes.

How do we write reproductions that work every time?

Here's our practical recipe for a reproduction that gets the bug fixed:

• Start from a clean state: specify the exact starting condition (e.g., "freshly installed database with seed data").
• List every input, including data values, file names, and user roles.
• Specify the environment precisely: browser, version, OS, screen resolution, network conditions.
• Include the exact sequence of actions, with any timing or repetition (e.g., "click the button 3 times within 2 seconds").
• State what you expected to happen and what actually happened — a screenshot or video is gold.

This is not just our opinion; it's aligned with how professional testers think. The NIST ACTS tool, for instance, is designed to generate test plans that cover combinations of two to six variables, precisely because those are the ones that trigger failures (NIST). If you apply that logic to reproduction, you'll ask yourself: "What combinations of conditions might matter here?" and then include those in your steps.

What about bugs that just won't reproduce?

Even with the best recipe, some bugs are stubborn. They might depend on rare timing or a specific user's data that you can't easily capture. In those cases, what do you do? First, don't give up. Try to isolate the variables. Change one thing at a time and see if the bug appears. Keep a log of what you've tried. This is essentially manual combinatorial testing.

Sometimes, you need to add instrumentation to the code to capture more information. That's not a failure of reproduction; it's a deeper investigation. The NVD, for example, doesn't test vulnerabilities itself; it relies on vendors and researchers to provide information (NVD). In the same way, you might need to rely on the developer to add temporary logging to a suspicious area.

But there's a limit. If after reasonable effort you can't reproduce it, you need to be honest. The bug tracker has a state for that: "Non-Reproducible" (Atlassian). That's not a shameful dead end; it's a signal that more information is needed. But the goal should be to avoid that state by writing a reproduction that works the first time.

What I'd actually do

Here's my bottom line: I'd make reproduction a first-class citizen in your bug tracking process. I'd require every bug report to include a "Reproduction Recipe" that follows the structure above, and I'd enforce it with a template. When a bug comes in without a clear recipe, I'd send it back immediately — not as a punishment, but as a way to save everyone's time.

I'd also invest in tools that help capture context automatically. Screenshots and video recordings are great, but even better are tools that record the exact user actions and environment. Many modern bug trackers integrate with such tools. But even without them, you can train your team to be disciplined.

Finally, I'd use the NIST insight as a mantra: "Most failures are caused by two to six interacting variables." When you're writing a reproduction, ask yourself: "What are the two to six variables that might interact to cause this?" and make sure you've controlled them. That single question will transform your bug reports from vague complaints into actionable recipes.

In the end, a bug report is only as good as its reproduction. Make your reproductions so precise that a developer can run them blindfolded. Your developers will thank you, your bug tracker will be cleaner, and your software will be better.

Sources

• Atlassian (bug life cycle) - https://www.atlassian.com/software/jira/guides/bug-tracking/bug-life-cycle
• NIST (2010 combination testing news release) - https://www.nist.gov/news-events/news/2010/11/updated-nist-software-uses-combination-testing-catch-bugs-fast-and-easy
• NVD (National Vulnerability Database) - https://nvd.nist.gov/general
• NIST (RTI 2002, Economic Impacts of Inadequate Infrastructure for Software Testing) - https://www.nist.gov/system/files/documents/director/planning/report02-3.pdf