In the world of software development, acceptance criteria play a crucial role in ensuring that the end product meets the client’s expectations. Traditionally, these criteria, or test cases as they are called, are developed by project managers, product owners, or QA testers. However, this manual approach has its limitations – it can be time-consuming, inefficient, and prone to human bias and error.
In fact, 35% of companies cite manual testing as the most time-consuming activity within a test cycle, with requirement changes (46%) and lack of time (39%) being the greatest barriers, according to research from Perfecto and Katalon. What’s more, manual testing is primarily focused on functional use cases, i.e., how many tasks were completed within a certain time frame, which is only one criterion that does not fully address the true measure of software quality over time.
AI offers new approaches that have the potential to revolutionize the way we define acceptance criteria and measure software quality. Here, we explore how AI can enable better acceptance criteria for software development, and what benefits it can bring to CTOs, CIOs, CFOs, and software developers.
Why traditional acceptance criteria fall short
Traditional acceptance criteria are often developed based on subjective human judgment, and they may not always reflect the true needs and expectations of end users or stakeholders. Further, they can be ambiguous, contradictory, or incomplete, leading to misunderstandings, disputes, delays, or even project failures.
Moreover, traditional acceptance criteria cannot adequately cope with the growing complexity and diversity of modern software applications, which often have multiple features, integrations, and dependencies.
How AI can improve acceptance criteria
Here are three areas in which AI can improve acceptance criteria for software development.
1. Functionality
AI-based tools can provide a more objective, comprehensive, and accurate approach to developing acceptance criteria for functionality. Machine learning algorithms can analyze large datasets of user feedback, bug reports, and usage patterns to identify the most critical features and scenarios that the software should support.
Natural language processing (NLP) techniques can help to translate human requirements into machine-readable formats that can be easily validated and tested. AI-driven testing tools can automate the verification process, thereby reducing the risk of human error and increasing the speed and coverage of tests.
2. Cost/Total Cost of Ownership
AI can help to optimize the cost-related acceptance criteria for software development. For example, AI-based forecasting models can predict the future resource utilization and cost of different software development scenarios, based on historical data and statistical analysis. This can help CIOs and CFOs make more informed decisions about resource allocation, budgeting, and pricing.
Further, AI can enable better Total Cost of Ownership (TCO) analysis, which takes into account not only the direct costs of developing and deploying software, but also the indirect costs of maintaining, upgrading, and retiring it over its lifecycle. By evaluating the long-term impacts of software design choices and maintenance strategies, AI can help to minimize TCO and maximize ROI.
3. Efficiency
In my post for Forbes, I talked about the importance of efficiency as a necessary acceptance criterion for the cloud:
Independent of whether a company chooses to house their operations in the cloud or on- premise, efficiency should be the second most important acceptance criterion following productivity. For an organization to be truly successful in the long term, they need to instrument and drive efficiency from the top down.
This measure should also consider the efficiency of vendor applications. Even without ownership of the source code developed by vendors, companies can update the indexing strategies or leverage technologies like data compression and other features to increase the efficiency of the code. There will inevitably be code that the vendor should update, in which case creating a feedback loop with vendors should be integral to your process for measuring efficiency.
AI can improve the efficiency of software development by automating various tasks, such as code analysis, optimization, refactoring, and deployment. This can help developers focus on the most critical and creative aspects of their work, while reducing the risk of errors, bugs, and technical debt.
Further, AI solutions such as Microsoft Copilot can provide real-time feedback and recommendations during the coding process, helping developers to improve their skills and productivity. By using AI-based tools for acceptance criteria, developers can also ensure that their code meets the desired performance, scalability, security, and reliability standards.
AI has the potential to enable better acceptance criteria for software development, and to bring significant benefits to CTOs, CIOs, CFOs, and software developers. By analyzing large datasets, automating tasks, providing objective feedback, and optimizing costs, AI can help to create more accurate, comprehensive, and efficient acceptance criteria for functionality, cost, and efficiency.
However, the implementation of AI-based tools and techniques requires careful planning, testing, and validation, to ensure that they do not introduce new biases, errors, or security risks. Therefore, organizations should approach AI-based acceptance criteria with a learning mindset, and involve all stakeholders in the process, from end-users to developers, to ensure its success and sustainability.