AI testing does not end at go-live
AI Testing
AI systems cannot be tested like conventional software and signed off once.
Their outputs are probabilistic, their environment changes, and acceptable performance depends on the use case and its risks.
Testing therefore has to establish what acceptable performance means, test it independently, and keep validating it throughout operation.
Our approach is informed by established AI testing and risk-management practice, including ISO/IEC TS 42119-2:2025, ISO/IEC 25059, ISO/IEC 23894, and the NIST AI Risk Management Framework.
Four principles of AI testing
Define success before you build
Testing starts with the use case, not the finished system. Define what the AI needs to achieve, how reliably it has to achieve it, and which failures are unacceptable.
Build evidence, not demos
A few good outputs show that an AI can work. They do not show that it works reliably. A proper test architecture separates development from independent validation and deliberately challenges the system beyond the cases it was built around.
Release is a baseline
Go-live establishes how the system performed at release. It does not prove that it will continue to perform that way. Regression testing catches deliberate changes. Recurring validation catches the changes nobody deliberately made.
Test the tester
Automation makes continuous testing possible. But the AI doing the testing has to be tested too. Human review provides an independent check on the evaluator itself.
AI rarely has a simple right-or-wrong output.
Testing therefore starts by translating the intended use and its risks into measurable acceptance criteria: required performance, acceptable error rates, failure behaviour and, where appropriate, probabilistic thresholds.
The depth of testing should follow the consequences of failure. An internal knowledge assistant and a system supporting consequential decisions should not face the same test regime.
This risk-based approach is central to ISO/IEC TS 42119-2. ISO/IEC 25059 provides a quality model for AI systems, while ISO/IEC 23894 addresses AI-specific risk management.
If success cannot be defined, it cannot be tested.
Without a definition of success
Testing falls back on impressions. Results get argued rather than measured, and “good enough” is settled after the fact.
Test depth also stops tracking risk: a low-stakes assistant and a system behind consequential decisions end up with the same shallow check.
Development teams need tests they can see and use continuously.
But once a system has repeatedly been optimised against the same cases, those cases become weaker evidence of its actual performance.
A robust test architecture therefore separates:
- Development tests used to improve the solution.
- Independent validation kept outside the optimisation loop.
- Regression tests covering capabilities that must not deteriorate.
- Stress and risk tests covering ambiguity, missing information, unusual inputs, adversarial behaviour and application-specific failure modes.
Because AI output is variable, important cases also need repetition and variation.
The question is not: can the system give the right answer?
It is: does it do so reliably enough for the intended use?
What a demo cannot show
Repeated optimisation against the same cases turns them into part of the build. They stop being evidence of anything.
Without validation kept outside that loop, a convincing demonstration and a reliable system look identical from the outside.
AI systems can change without a conventional software release.
Models are updated. Knowledge bases change. Retrieval produces different context. Prompts and tools evolve. External services change. Users find new ways of interacting with the system.
Known changes should trigger regression testing.
But testing also needs to run independently of releases to establish that production performance remains within the accepted range.
The NIST AI Risk Management Framework calls for testing before deployment and regularly during operation. ISO/IEC TS 42119-2 likewise treats AI testing as a lifecycle activity and recognises continuous testing as a response to changing system behaviour.
Production is not where testing stops. It is where continuous validation begins.
Why a baseline decays
Go-live records how the system behaved on one day, with one version of every model, prompt, index and service behind it.
All of those keep moving, and most move without a release. Recurring validation is what keeps the baseline describing the system that is actually in production.
Human in the loop does not mean asking people to inspect every AI output.
Humans define evaluation criteria, establish reference cases, investigate material failures and review situations where context or expert judgement matters.
Automation provides the scale required to execute those tests continuously.
But the evaluator also needs oversight.
Cases identified as uncertain or problematic should be escalated for review. Just as importantly, random samples of outputs judged to be successful should also be reviewed by humans.
That distinction matters. Reviewing flagged cases tells us whether the evaluator’s alerts are useful. Random sampling can reveal what it failed to detect.
Human review therefore does more than supervise the AI being tested. It continuously validates the testing system itself.
Why random samples
Reviewing flagged cases shows whether the evaluator’s alerts are useful. It says nothing about what the evaluator never flagged.
Sampling the outputs it judged successful is the only way to see that blind spot — so the tester is measured as deliberately as the system under test.
Automated testing: a stable judge for a moving target
Continuous AI testing requires automated evaluation.
But if the evaluator itself changes outside your control, it becomes difficult to tell whether a changed test result reflects the system under test — or the judge.
The 9senses automated testing suite uses a dedicated, calibrated and version-controlled evaluator that can be deployed locally or within a controlled client environment.
Its model version, weights and evaluation configuration remain fixed until they are deliberately changed and revalidated.
That creates a stable benchmark against which changes in the system under test can actually be measured.
Controlled evaluation
The current evaluator is based on Mistral and can be used across GenAI and other deep-learning applications where outputs can be assessed against defined criteria.
The suite can continuously:
- execute regression and validation suites;
- evaluate outputs against use-case-specific acceptance criteria;
- identify deviations and emerging failure patterns;
- surface edge cases for expert review;
- route uncertain cases into human validation;
- send random samples for independent review; and
- track performance against a known evaluator version over time.
Because the evaluator can run inside the testing environment, sensitive test data does not have to be sent to an external frontier-model provider for evaluation.
The evaluator is part of the test architecture
A fixed evaluator does not mean an unquestioned evaluator.
Human-reviewed reference cases, edge-case escalation and random sampling are used to validate how the evaluator itself performs.
When a new model version, new weights or a material configuration change is introduced, the evaluator can be recalibrated and validated before becoming the new testing baseline.
The evaluator does not undergo uncontrolled model drift. When it changes, that change is tested too.
Automation provides the scale. Human review provides the control.
9senses Automated Testing is currently in beta.