AI QA: Testing Your App with Artificial Intelligence
AI QA testing with gstack's Test phase: use the QA Lead and Performance Engineer roles to find bugs before users do. Step-by-step guide.
AI QA — using artificial intelligence to test your application — is now one of the most practical things a solo developer or small team can do before shipping. With gstack's Test phase, you run your feature through a QA Lead role that writes test cases, hunts edge cases, and stress-tests your assumptions, all before a single user clicks anything. This guide shows you exactly how to do it.
What AI QA Actually Does (and Doesn't Do)#
Let's be clear about the scope. AI QA isn't a replacement for automated test suites. It's a layer of structured adversarial thinking that catches problems your happy-path tests miss.
The QA Lead role in gstack's 18 specialist roles approaches your feature from a user-hostile perspective: what inputs will break this? What happens when the network drops mid-operation? What does a non-technical user do when they encounter an error?
Automated tests verify what you expected. AI QA finds what you didn't expect.
The combination — AI QA feeding into a written test suite — is what produces genuinely robust software. And after you've done code review with AI, QA is the natural next step in the gstack workflow.
Step 1: Document What You Just Built#
The QA Lead needs to understand what the feature is supposed to do before they can test it. Start with a feature brief:
Feature: [Name]
What it does: [one paragraph]
Entry points: [where users access it]
Key user flows:
1. [happy path]
2. [secondary path]
3. [error path]
Preconditions: [what must be true for it to work]
Dependencies: [other features, APIs, services it relies on]
This brief is also useful for human QA, documentation, and support. Write it once, use it everywhere.
Step 2: Generate the Test Case Matrix#
Here's the prompt for the QA Lead role:
You are a QA Lead for a production SaaS application.
Given this feature brief:
[paste brief]
Generate a comprehensive test case matrix covering:
1. Happy path scenarios (at least 3)
2. Edge cases (at least 5)
3. Error conditions (at least 5)
4. Boundary value tests
5. Concurrency scenarios (if applicable)
6. Security test cases (invalid auth, IDOR, etc.)
Format as a table: | Test ID | Description | Input | Expected Output | Priority (P0/P1/P2) |
The priority column is important. P0 = ship-blocking, P1 = should fix before release, P2 = known limitation. This forces a triage decision on each test case rather than treating all bugs as equal.
Step 3: Walk Through the Matrix Manually#
Before writing any automated tests, walk through the P0 and P1 test cases manually in your dev environment. This sounds tedious but takes 20-30 minutes and catches the issues that automated tests would have been written around.
Document what you find:
## QA Session: [Feature Name] - [Date]
### P0 Tests
| ID | Status | Notes |
|----|--------|-------|
| TC-001 | PASS | |
| TC-002 | FAIL | Error: null pointer when email is empty |
| TC-003 | PASS | |
### Bugs Found
- **BUG-001** (P0): Null pointer exception when email field is empty on submit
- Steps to reproduce: Leave email blank, click submit
- Expected: Validation error
- Actual: 500 errorThis document becomes your bug tracking for the feature. Store it in your workspace.
Step 4: Write the Tests#
Now that you know what to test, ask the AI to write the actual test code. This is where the QA Lead hands off to the implementation:
You are writing tests for a [language/framework] application.
Based on this test case matrix: [paste matrix]
And this implementation: [paste relevant code]
Write [Jest/Pytest/RSpec/etc.] tests covering all P0 and P1 cases.
Include:
- Setup and teardown
- Meaningful assertion messages
- Comments explaining non-obvious test logic
The key: you've already validated the test cases manually. The AI is writing tests for known behaviors, not inventing scenarios. This produces dramatically better test code than asking AI to "write tests for this function."
Step 5: Run the Performance Engineer Review#
Performance bugs are QA bugs. The Performance Engineer role in gstack looks specifically at:
You are a Performance Engineer reviewing this feature for production performance.
Feature description: [paste]
Implementation: [paste code]
Analyze:
1. Database query patterns — N+1s, missing indexes, full table scans
2. Memory allocation patterns — are we holding too much in memory?
3. Network round trips — can any be batched or eliminated?
4. Caching opportunities — what can be cached and for how long?
5. Worst-case scenario — what happens at 10x expected load?
Output specific, actionable findings with severity (HIGH/MED/LOW).
Performance issues that make it to production are almost always issues that were visible in the code before they were visible in production. The Performance Engineer role makes them visible earlier.
Step 6: Stress Test Your Error Handling#
One of the most common QA failures is inadequate error handling. Here's a targeted prompt:
You are a chaos engineer testing the resilience of this feature.
Implementation: [paste code]
External dependencies: [list APIs, databases, services]
For each external dependency, describe what happens if:
1. It returns a 500 error
2. It times out after 30 seconds
3. It returns unexpected data format
4. It returns empty/null response
5. The connection is dropped mid-request
Is the error handling appropriate for each case? What would the user experience?
This systematically surfaces every place where you've written catch (e) { console.log(e) } instead of actually handling the error.
Step 7: Run the Accessibility QA Check#
If your feature has UI, accessibility is a QA concern, not just a design concern:
You are an accessibility QA specialist.
Review this UI implementation: [paste HTML/component code]
Check for:
1. ARIA labels on interactive elements
2. Keyboard navigation completeness
3. Focus management after modal/dialog interactions
4. Color contrast (describe what you see, flag potential issues)
5. Screen reader announcement for dynamic content changes
6. Form validation error announcement
7. Timeout/session expiry notification
Output: PASS/FAIL for each with specific fix instructions for failures.
WCAG violations are bugs. Treat them as bugs.
Step 8: Write the QA Sign-Off#
Before shipping, document that QA was completed:
## QA Sign-Off: [Feature Name]
Date: [date]
Build: [commit hash or version]
### Coverage
- [ ] All P0 test cases: PASS
- [ ] All P1 test cases: PASS (or documented exceptions)
- [ ] Automated tests written for P0/P1
- [ ] Performance review: completed
- [ ] Error handling review: completed
- [ ] Accessibility check: completed (if UI feature)
### Known Issues (P2 / Won't Fix)
[list any known limitations accepted before ship]
### Test Commands
```bash
npm test -- --testPathPattern=feature-nameQA complete. Ready for Ship phase.
This document goes into your PR and into your workspace. It's how you prove that QA happened, not just that code was written.
## Integrating AI QA into Your Workflow
The full gstack flow — Think → Plan → Build → Review → **Test** → Ship → Reflect — has QA happening *after* code review but *before* ship. This sequencing matters.
Code review with AI ([previous article](/blog/code-review-with-ai)) catches implementation bugs. QA catches behavioral bugs — the gap between "the code does what it says" and "the code does what users need."
Running them in sequence means by the time you hit Ship, you've had:
1. Staff Engineer review the implementation correctness
2. Security Auditor check for vulnerabilities
3. QA Lead generate and walk through test cases
4. Performance Engineer check for bottlenecks
5. Automated tests covering the critical paths
That's more thorough than most well-resourced teams achieve. And it's achievable by a solo developer working on a laptop.
## Using This with DenchClaw
[DenchClaw](/blog/what-is-denchclaw) stores your QA documents, test case matrices, and sign-off reports as workspace documents — searchable, linked to the features they cover, and accessible to the whole team.
When someone asks "was this edge case tested?" you can actually answer the question. That's the difference between a mature development process and "we test in production."
---
## FAQ
**Can AI actually find real bugs, or is it just generating suggestions?**
In practice, yes — especially for edge cases and error handling. The AI isn't running your code, but it's reading it with a specific adversarial frame. The bugs it finds are real; whether they're in your specific implementation depends on how much context you provide.
**What if I don't have time to write automated tests?**
Prioritize. P0 tests should have automated coverage before ship. P1 before next sprint. P2 as time permits. Even 5 automated tests covering the critical paths are better than none.
**How do I handle the AI generating test cases for behavior I haven't implemented?**
That's valuable signal. If the AI generates a test case that doesn't apply yet, either implement the behavior or document it as a known limitation. Don't just delete it.
**Should I use AI QA for bug fixes, or just new features?**
Both. For bug fixes, specifically ask the QA Lead to generate regression tests that would catch this bug if it returns. This is how you prevent the same bug from shipping twice.
**What models work best for QA?**
Any capable model works for test case generation. For code-level performance analysis, models with good code understanding (Claude Sonnet, GPT-4o) give better results. The role framing matters more than the model choice.
---
*Ready to try DenchClaw? Install in one command: `npx denchclaw`. [Full setup guide →](/blog/openclaw-crm-setup)*
