AI can generate test cases, write automation code, analyse logs and prepare reports in minutes.
That is a major advantage. But it raises a question that matters more than speed:
Can AI assure product quality?
Test cases, scripts and reports are outputs. They help us gather evidence, but they are not the final goal. The goal is a product that works correctly, protects its users and supports the business.
AI can accelerate testing. It cannot make quality automatic.
In Part 1, we examined how AI is changing traditional QA work. Part 2 looks at where AI can fall short and why core QA skills must complement it.
Testing activity is not product quality
A team may have hundreds of test cases, high automation coverage and a green execution report—and still release a poor product.
The tests may confirm the wrong assumptions, overlook an important risk or examine each feature without considering how the complete system behaves.
Generating more tests does not automatically answer:
- Did we understand the customer’s real need?
- Is an important business rule missing?
- Which failure would cause the greatest harm?
- Are the expected results correct?
- Does the feature work across the complete system?
- Is the evidence strong enough to release?
AI can help answer these questions, but not reliably without the right context and human judgement.
AI can produce testing artefacts. Quality professionals must decide whether they provide trustworthy evidence about the product.
Consider a simple money-transfer requirement
Suppose the requirement says:
A customer can transfer money to another account.
AI may quickly generate tests for valid and invalid accounts, sufficient and insufficient balances, transfer limits, confirmation messages and transaction history.
These are useful tests. They are also only a beginning.
An experienced tester may ask:
- What if the customer submits the transfer twice?
- Can simultaneous requests spend the same available balance?
- What if the debit succeeds but the credit fails?
- What happens if the network drops after submission?
- Is the daily limit applied to one transfer or the combined total?
- Can the recipient be changed by modifying the API request?
- What audit evidence and recovery process are required?
The difference is not typing speed. It is the ability to understand the product, identify consequential risks and investigate how the system could fail.
Where AI can fall short
AI’s output depends on the information it receives and how the task is framed. Several limitations matter directly to QA.
| Limitation | Why it matters |
|---|---|
| Hidden assumptions | When information is missing, AI may fill the gap with a plausible but unverified rule. |
| Illusion of coverage | Differently worded tests may exercise the same condition and make a weak suite appear thorough. |
| Missing context | AI does not automatically know undocumented rules, previous incidents, architecture or regulatory obligations. |
| Incorrect expected results | If the source requirement is incomplete or wrong, AI can reproduce the weakness confidently. |
| Shallow failure analysis | Timing, concurrency, retries, state transitions and recovery across systems require deeper context. |
| Fragile automation | Generated code may run while containing brittle selectors, fixed waits, weak assertions or unsuitable design patterns. |
Clear language and professional formatting are not proof of correctness. “The script runs” is not the same as “the automation is reliable.”
These limitations are not reasons to reject AI. They are reasons to apply professional judgement.
Faster generation makes evaluation more important
When content becomes cheap to generate, evaluating it becomes the harder task.
The Stack Overflow 2025 Developer Survey reflects this tension. Although AI use is widespread, 46% of respondents distrust the accuracy of AI tools, compared with 33% who trust them. The most frequently reported frustration is output that is “almost right, but not quite.”
An obviously wrong answer is easy to reject. A plausible answer containing one hidden mistake is more dangerous because it may survive a quick review.
QA professionals must evaluate AI output for:
- Accuracy: Is it correct?
- Coverage: What important behaviour or risk is missing?
- Relevance: Does each test contribute useful evidence?
- Traceability: Which requirement, rule or risk supports it?
- Maintainability: Does generated code fit the project?
- Safety: Has sensitive information been exposed or mishandled?
- Value: Does it save effort after review and correction?
The solution is neither blind acceptance nor exhaustive manual checking. It is a disciplined method for deciding what requires verification and how to verify it.
The core QA skills that must complement AI
As generation becomes easier, the skills that guide and evaluate it become more valuable.
Product and domain knowledge
AI understands common patterns. It does not automatically know which exception previously cost the company money, which undocumented rule operations teams depend on or what customers consider unacceptable.
For example, AI may generate standard tests for cancelling an insurance policy. An experienced tester may know that the rules change after a claim is submitted or while a payment is being processed.
That knowledge changes what must be tested.
Risk-based thinking
AI can generate hundreds of tests. A team rarely has unlimited time, environments or data to execute everything.
Someone must decide which failures would cause the greatest damage, which journeys are business-critical and what must be tested first. If only five tests can run before release, selecting the right five is more valuable than generating another hundred.
AI can suggest priorities. A QA professional must validate whether they reflect the real business risk.
Critical evaluation
AI output can be detailed and confident while still being incomplete or incorrect.
Quality professionals must detect invented requirements, unsupported assumptions, incorrect expected results, duplicate coverage, missing scenarios and unsafe recommendations.
The skill is not to accept or reject AI automatically. It is to know what evidence would confirm or disprove its output.
Exploratory investigation
Important defects are often discovered when a tester observes something unexpected, asks a new question and changes direction.
One unusual response may lead to another user journey, a different API call or a deeper investigation. The next step cannot always be written in advance.
AI can suggest possibilities and analyse evidence. The tester supplies the curiosity, purpose and judgement that guide the investigation.
System thinking
A feature may work on one screen and still fail across the complete system.
A payment may appear successful while the account is debited twice, a downstream service misses the event, a retry creates a duplicate order or recovery fails after a partial outage.
Quality exists across user interfaces, APIs, databases, queues, permissions, external services and operational processes. AI can analyse individual components; system thinking reveals which connections and consequences matter.
The value of QA is shifting
AI does not remove the need for core QA skills. It changes where those skills create the greatest value.
| Earlier emphasis | Growing emphasis |
|---|---|
| Producing more test cases | Selecting meaningful coverage |
| Following documented steps | Investigating unexpected behaviour |
| Checking isolated features | Understanding end-to-end risk |
| Writing routine automation code | Evaluating and improving generated code |
| Reporting failures | Explaining their customer and business impact |
| Accepting requirements as written | Challenging assumptions and ambiguity |
The tester’s contribution moves from producing more artefacts to making better quality decisions.
When AI can generate more, QA professionals must become better at deciding what matters.
AI capability and QA expertise must work together
The choice is not between AI and human testing. The stronger model combines their different strengths.
AI can generate initial ideas, examine requirements, analyse large volumes of results, suggest failure scenarios and review existing tests or code.
QA professionals must supply context, verify assumptions, prioritise risks, investigate beyond predefined tests and judge whether the evidence can be trusted.
A practical partnership has four steps:
- Provide relevant context. Include business rules, architecture, existing coverage, previous defects and quality expectations.
- Use AI to generate and challenge. Ask it to find ambiguity, question assumptions and identify gaps—not only to produce content.
- Apply QA judgement. Verify the output, investigate the system and add knowledge AI does not possess.
- Decide using evidence. Determine whether important risks have been examined and whether the remaining risk is acceptable.
AI provides speed and scale. QA expertise provides direction, judgement and accountability.
AI can accelerate testing. Core QA skills turn that acceleration into product quality.
Product quality remains the goal
AI can write tests, generate automation, analyse failures and prepare reports. But output volume alone cannot tell us whether the product solves the right problem, behaves correctly under difficult conditions or can be trusted by customers and the business.
AI can support quality decisions. It cannot own their consequences.
The future does not belong to QA professionals who ignore AI, or to teams that accept everything it produces. It belongs to quality professionals who combine practical AI capability with product knowledge, risk thinking, critical evaluation, exploratory investigation and system thinking.
Quality expertise combined with AI capability offers the strongest path to product quality.
The next change is bigger than AI assisting traditional testing. Software itself is beginning to interpret goals, make decisions, use tools and take actions.
That creates a new question—and a new frontier for QA.
In Part 3, we explore what it means to test AI agents.