Can AI play a role in Quality Engineering
AI can significantly help and improve the Quality Engineering space with numerous benefits. All these benefits can be achieved with proper planning across each phase of the STLC (Software Testing Life Cycle). AI models can be trained to understand business problems, analyze the requirements, optimize the test authoring/documentation and reduce the defect leakage to the production, thereby increasing the overall product quality. Apart from facilitating automation, AI Models can allow for deeper data analysis and prediction levels by producing a report that users with a technical background can easily consume. This blog explores how AI can be utilized across various phases of the software testing life cycle, uncover the challenges faced by clients and how AI can help address those challenges.
Challenges – What we hear from the Customers
- Too Less or Too Much Testing: Governance around creating and maintaining a test pack is mostly missing in an organization’s Testing Methodology. Because of this, the test pack gets piled up with new test cases/artifacts without checking for relevance and even for duplicity. Lack of traceability often leads to an increase in testing efforts and rework.
- Quality Deterioration impacting the total Cost of Quality: Test Strategies should be targeted to focus on Shift Left testing for defect prevention early in the lifecycle.
- Less Agile: Need to increase or incorporate in-sprint automation. Need to increase/maximize test effectiveness and increase velocity. Lack of Shift Left Approach in testing without continuous testing design to deployment.
How AI can Help to Address the Above Challenges
Requirements Phase
One of the major reasons for defect leakage is the quality of requirements captured. Let us, for our discussion, divide “Requirements” into three categories: High, Medium and Low. AI Model can be trained to classify the requirements based on the above three elements and ascertain the quality of the requirements. Results from the model can be used to analyze the health of the “Requirements” and decision/action can be made/planned out to take corrective action.
Test Planning / Preparation Phase
AI Model can be used to analyze the regression suite over last few releases to identify the defects identified across particular sets of functionalities and related test cases. This will help in finalizing and boiling down to a precise set of regression test suites that would be helpful for a particular feature release. Also, if there are no defects identified from a particular set of regression test suites – they can be marked as secondary during a hot fix. This process will help in “Prioritizing” Test Cases. Another aspect in which AI / Gen AI can help is identifying the right sets of data sets and test cases for Regression, E-2-E (end to end) Testing and Smoke Tests.
Execution Phase
AI can play a major role during the execution phase by not only reducing the “Time to Market” but also during the ongoing “Maintenance Phase”. There are two main aspects of AI that can play an important role during the execution phase:
- Self-Healing / Auto Healing
- Object Learning
Self-Healing
Let us assume that the testing team recorded few scripts. Before the execution phase started there were few changes in the application like: (1) the DOM of the application changed, (2) labels got changed, (3) position of the elements got changed, (4) a new field got added or (5) internal properties of the object changes. Typically, any of these changes will call for additional changes in the existing script there by increasing the maintenance efforts. With the Self-Healing AI Algorithm this will be taken care of automatically with no additional effort for script correction/maintenance efforts.
The way it works is: During recording, each element in the page will get captured as an object with multiple locators. Consider locator as object identifiers. During execution, if any of the locators is not found, the Agent (Component that operates on the testing scripts and perform specific function) looks for another locator. In addition to these locators, the Agent also captures the object image. Object image will come in handy if none of the locator works.
Object Learning
Let us consider a scenario in which the application undergoes a change resulting in the addition of objects which were not present during the test recording. In this case, AI Algorithm will make the Agent smart enough to identify additional properties of the additional element and a corresponding locator gets constructed and placed in the object repository. This will work in conjunction with the Self-Healing Algorithm to ensure the execution takes place smoothly without any additional maintenance effort.
Analysis Phase
AI Models can play a vital role in the Analysis Phase. Let us consider you have executed around thousands of test cases and the testing team needs to categorize the defects based on type, severity, dependency on other defects and then finally assign it to the right team for closure. AI Algorithm can help in analyzing and categorizing the defects whether it is due to application, automation framework, product, requirements or due to data / environment. This analysis can tremendously reduce the time spent by the team to go through each defect, classify and then further route it to the right team. Also, the AI Model can be trained to detect dependencies between defects, and analyze the time it took to close similar defects.
Few Examples of AI-driven Quality Engineering
AI Driven Voice Recognition
Voice command is 3 to 5 times faster than a keyboard command. AI Models can be trained to recognize the voice command from the user and perform the desired actions. Model can be fine tuned based on the demographics and the user profile. Apart from improving the speed of delivery, this approach can also help improve the overall customer experience. This feature can be helpful when writing Accessibility test cases for users having motor disabilities. Also, it can aid in logging defects on a runtime with minimal effort.
AI Driven Requirements Generator
One of the use cases that we discussed above was requirements classification to understand the health of requirement. In addition to this AI Model can help in generating new requirements based on the training data (requirements). For example – if the AI Model is given the inputs for UI/UX requirements – it can also generate Accessibility requirements. This in turn will help in increasing the test coverage and also reduce the time required for planning/gathering of additional requirements.
Benefits of AI in Quality Engineering
From the above examples, we can conclude that AI can be utilized throughout the software testing lifecycle with multiple benefits across quality, cost, and time to market.
- Quality: AI can help in identifying the percentages of failures against the number of test executions. This can help us concentrating in a focused approach and prioritizing the test case execution for optimal results.
- Time to Market: Optimized test suite can reduce time-to-market by 55%. Overall, the reduction in the test execution cycle will be about 30%.
- Cost: With optimal test cases and prioritization done, the number of test runs will drastically reduce to 60%. The overall maintenance required will be reduced.
Conclusion
As we can see from above, using AI in Quality Engineering has multifold benefits. AI can be a powerful tool right from the “Requirements” phase going up to the “Analysis” phase, helping clients to reap benefits across each testing phase. Incorporating AI can help in having an optimized test repository that can not only reduce the ongoing maintenance effort but also increase the time to market. The benefits of AI model can be equally leveraged during analysis and risk assessment/mitigation strategy.
Discover how AI for software testing can optimize your testing processes.
