One of the important aspects of Test strategy is to define the defect tracking and reporting mechanism. The software defect is defined as a problem with functionality, display, reports, or output where there is a deviation from the system specification.
While managing the project, project manager has limited development resources, which are assigned for fixing defects, hence it is very important to assign the priority to defects to define schedule for fixing those defects. When a tester logs the defect, he/she defines the severity for the same. Severity of the defect is defined based on level of issues it causes in the system in testing.Tester can select severity on the basis of levels stated below, or they can define their own standard for severity, which will be used by testing team –
1. Blocker – defect is marked as a blocker by the tester when he/she is:
i. Unable to start the application
ii. Application crashes after initialization
iii. Data is corrupt. User information is lost or damaged
iv. A specific business user/role is completely impeded from any productive work with the system
2. Critical – defect is marked as critical by the tester when:
i. Specific module of the application is not functioning as expected
ii. Specific modules return incorrect values
iii. A business user/roles is impeded in a limited fashion.
3. Major – defect is marked as major by the tester when:
i. A particular piece of functionality works, but should be improved or doesn't meet requirements
ii. A business user/role may be inconvenienced, but is not significantly impeded due to workarounds available.
4. Minor - Defect is marked as minor by the tester when:
i. Minor feature of the application is working incorrectly
ii. A business user/role is not significantly inconvenienced, but experience could be improved.
5. Trivial - Defect is marked as trivial by the tester for:
i. Cosmetic issues in the application, such as spacing or alignment issue which is not very obvious.
Once the defect is reported, triaging is performed. In the triage meeting a priority is assigned to allow the engineering team to concentrate on the defects that must be fixed first. There are four categories used for the prioritization process - P1, P2, P3, and P4.As severity defines implication of a defect on the system under testing, priority of a defect defines the importance of a defect to be fixed from business user perspective. Thus, bug prioritization is performed in Bug Triage meetings wherein all stakeholders are involved. It is very important to define priority of a defect in consent with all stakeholders. These meetings are conducted involving testing team, Development team, Business users, End Users. In certain scenarios, for example product development, Sales team and Finance team is also involved.
The priority definition lays out sequence in which defects need to be fixed by development team. All defects will be categorized into four categories and prioritized accordingly –
Priority P1- It is termed as show stopper and must be fixed ASAP.
All the defects which fall under the severity category of blocker and critical are usually assigned priority value of P1. The defects under priority value P1 should be fixed immediately. This covers issues that prevent execution of major Functionality, which has no workaround.
Priority P2- It is termed as critical and must be fixed prior to release.
All of the defects which fall under the severity category of major are usually assigned priority value of P2. The defects under priority value P2 should be fixed prior to release. This covers issues that prevent execution of major functionality but has a workaround.
Priority P3- It is termed as moderate and should not delay the release.
All of the defects which fall under the severity category of minor are usually assigned priority value of P3. The defects under priority value P3 can be released as patches after first release and should not cause schedule delay. This covers Usability Issues that prevents execution of sporadically used or minor functionality. Further, application defects caused by unusual usage or spelling and translation errors are priortized as P3.
Priority P4- It is termed as cosmetic and can be included in subsequent releases.
All the defects which fall under the severity category of trivial are usually assigned priority value of P4. The defects under priority value P4 is the lowest priority and lowest return on fix defects. Based on agreement with stakeholders these defects can either be marked as won’t fix or can be scheduled for future releases.
Once the priority is defined, response and turnaround time is defined for each priority type. Each organization can have different expectations for turnaround time. The turnaround time helps in defining the next testing cycle and also helps in monitoring over-all progress of the release cycle.
As a last step, the exit criterion is defined, which defines a defect acceptance level against each priority. Usually as a best practice, the defect acceptance criteria should have – zero P1, zero P2, specified number of P3 based on size of application and unbounded limit for P4.
The defined exit criteria must be achieved before the software can be recommended for turnover for Production Deployment. The aforementioned exit criteria must be met before a set of code or Sprint can advance to the next stage of testing. For example, there must be zero P1, no more P2, and specified numbers of P3 defects before a set of code can move from the development environment to the quality assurance environment. This applies to each QA stage: System Testing, User Acceptance, Beta and Production.
|< Prev||Next >|