Sometimes the apparent simplicity and recurrence of reporting bugs make us overlook its importance, reporting poor, naive or misguided bugs, causing waste of time of other testers, developers and managers. The simple task of reporting a bug is very important and should be guided by standards and guidelines to efficiently carry out all the effort required for its discovery (test planning, test cases and test execution). This article discuss a bit about this task, and proposes some standards and guidelines to help in the process, that should be seen as suggestions and by no means absolute.

First of all, the bug reporting activity should be directly related to the software testing activity, which in turn is directly related to the software development life cycle model. The more formal the testing activity is, the better to report bugs. Also, as soon as the bug is reported, the better.

When you're reporting a bug you need to remember that the developer first needs to understand it, then try to reproduce it, and just then try to fix it. Don't assume anything, be as most specific and clear as possible. Clearly describe the facts that happened, the expected results and the received results. Also, if you find appropriate you can provide some personal thoughts on what may have caused the bug, but make a clear distinction between facts and thoughts. Another important thing is that you should report one bug per report. Generally, it's not a good idea to group bugs together due to implications like its correction, organization and traceability.

Besides a clear description, the reproducibility of a bug is very important. Sometimes it's easy to reproduce a bug, as it's easy to provide its steps. But sometimes it's hard to do it, and techniques to isolate the bug may be necessary, such as careful attention to each interaction, gathering information about the feature being tested and its dependencies, the hardware being used, attention for race condition problems, etc. The fact is that the steps to reproduce a bug is the most important part of the report. If the developer can't reproduce it, and thus see it, how can he fix it? But if it's impossible to reproduce a bug, and then provide its steps, it's still valuable to report it, but its description should be even better.

An important behavior when you're testing software and find a bug is to stop interacting with the software, then analyze and record what happened. Reporting a bug that occurred only because another previous bug has occurred is almost always useless and will waste time of your team. You need to identify and isolate the first bug, the one that can trigger an instability in the software and consequently other bugs, if it doesn't break the system entirely.

After you have a good description and the steps to reproduce it, you need to classify the bug according to its impact in the software, so it can be prioritized in the bug queue to be fixed. According to Patton (231), a "common method for doing this is to give your bugs a severity and a priority level". Severity indicates the impact of the bug in the software, and priority indicates the urgency to make the fix.

Following is a list that Patton (231-232) provides just as an example of such classification:

Severity

1. System crash, data loss, data corruption, security breach
2. Operational error, wrong result, loss of functionality
3. Minor problem, misspelling, UI layout, rare occurrence
4. Suggestion

Priority

1. Immediate fix, blocks further testing, very visible
2. Must fix before the product is released
3. Should fix when time permits
4. Would like to fix but the product can be released as is

Although you can think that it's obvious that the most severe bug should be prioritized first, this is not always the case. For example, a very severe and hard to fix bug (e.g. severity 1) may only occur in a very rare scenario (e.g. priority 3), while two other silly, easy to fix bugs (e.g. severity 3) can cause a big impact if released (e.g. priority 1).

Some projects may require extra information about its bugs, and it can also change according to the tracking system used. Among the extra data, we can have a unique identifier, a title, a bug summary, date and time, the tester's name, hardware and software configuration, related test cases, etc. If any of these information is relevant it should be provided.

And speaking of tracking systems, it's clear that for a good bug management it's needed a good tracking system, and the tester (the bug reporter) should be concerned about it because it's also his responsibility to monitor the bug's life cycle, making sure it gets fixed.

Related posts

What is a software bug?
What is software testing?
Testing is not Quality Assurance (QA)
What is a product specification?

Interesting links

Bug writing guidelines - MDN
How to Report Bugs Effectively (by Simon Tatham)

Bibliography

Patton, Ron. Software Testing. 2nd ed. Sams, 2005.