I suggest a different approach: analyzing your risks
. All the risks in software development fall into two types: technical risks (the product will not work as intended) and business risks (when a technical failure may influence the business). Common risk factors for the software development projects are: 1. Business-related Quality risks:
2. Development-side Quality risks:
- Affect of the failure on company revenue
- Affect of the failure on the product churn rate
- Number of customers using the product day-to-day
- Number of other products using the output of the system
- Sophistication of the end users
- Subject-area quality standards compliance (e.g., medical or government software)
- Costs associated with late delivery
- Visibility of the product quality by senior management
- Unexpected Requirements changes
- Misestimating Workload
- Turnover of project team members
- Engineering compromises
- Fresh, unproven technologies
- Bad project management
- Test environments instability
- Team experience (specific toolset)
- Lack of communication or documentation
- Changes in the functionality with new releases
If technical and business risks on the project are low
, there is no big need in structured testing at all. The best you can do is to conduct some manual exploratory testing and review it with the stakeholders.
If your analysis indicates medium
or above average risks it's time to focus on unit testing. Also add some manual black-box testing including test analysis, design, review and support of the tests.
But if your risks are high
, make sure you have the following areas covered: 1. Control your manual testing:
2. Use certain amount of automation on multiple layers:
- Each test case is detailed and stored in the test management system, grouped in test suites.
- All test cases are reviewed and formally signed off by the Development and Product Management team representatives.
- All the manual testing is accounted in TMS so you always control what, when and how is tested.Whenever the issue is missed, you can analyze the reasons and identify underperforming engineers.
- Ad Hoc testing is allowed only for product managers and developers. QA is not allowed to test ad hoc.
- Whitebox tests (Unit, Integration, System level)
- Blackbox tests (UI and API logic)
- Performance test automation (Load and Volume tests)
3. Assess System Security on a regular basis:
Working approach: there is the principle of diverse half-measures
: apply a bunch of methods instead of investing 100% of your budget in manual testing. This will save you much costs.
Understanding the risks not only helps you to mitigate them, you will be more confident in the budget to spend for a certain area of the project.