In this article
Artificial intelligence (AI) now sits inside a lot of everyday business work: transcribing calls, summarising meetings, flagging compliance risks, monitoring customer interactions. In most organisations it isn’t replacing people, it’s helping them work faster and more consistently.
That shift raises a harder question once AI output begins influencing real business decisions. Being accurate most of the time stops being enough. Someone eventually has to prove the system is still working as intended, not just assume it.
What Is AI Assurance?
AI assurance is the practice of generating evidence that an AI system is reliable and behaving as intended. It answers the questions that come up the moment something goes wrong, or an auditor asks:
- Has this workflow been tested?
- What changed since the last check?
- Which model and prompt version produced this output?
- Did a person review it?
The UK government’s Department for Science, Innovation and Technology 1 describes assurance as the process of measuring, evaluating, and communicating whether an AI system can be trusted, the same underlying idea as a financial audit or a security certification, applied to AI.
The European Commission 2 takes a similar view from the regulatory side. Its framework for AI is risk-based: the higher the potential impact of a system, the stronger the testing, documentation, and oversight expected around it. Different starting points, same conclusion: trust in an AI system has to be demonstrated, not assumed.
Two other frameworks are worth knowing. The US NIST AI Risk Management Framework 3 organises AI risk management into four functions: govern, map, measure, and manage, which together rely on measuring and documenting AI system performance. ISO/IEC 42001 4, the first international, certifiable management system standard specifically for AI, and the OECD AI Principles 5 point the same direction: know how your AI is used, test it, and keep records.
Governance sets the rules, assurance proves you followed them
Governance without assurance is a policy nobody can verify. Assurance without governance is testing without a standard to test against. You need both.
| Discipline | Focus | Question it answers |
| AI Governance | Policies, accountability, organisational controls | How should AI be used responsibly here? |
| AI Assurance | Evidence, testing, verification | Can we show this system behaves as intended? |
In practice, organisations need a repeatable way to generate assurance evidence as AI models, prompts, and workflows change over time.
Where Argus Archive fits in
Organisations often discover, usually after an incident, that they can’t actually show how an AI-assisted decision was made. Which model produced it, whether a person reviewed it, whether the workflow had been tested against a case like this before. That gap is normally invisible until someone needs the answer and it doesn’t exist.
Learn how AI Assurance helps teams test changes, build evaluation datasets and maintain assurance evidence throughout the AI lifecycle.
Instead of testing a workflow with whatever examples are lying around, Argus Archive lets teams build evaluation sets; a fixed, repeatable set of test scenarios, run the workflow against it before and after any change, and keep a permanent record of how performance moved. Alongside that, it tracks which model and prompt version produced each output, who reviewed it, and the full version history of the workflow over time.
Evaluation sets versus benchmarks
These terms get used loosely, and the difference matters. A benchmark is usually a public, general-purpose test, built to measure how capable a model is in the abstract. An evaluation set is usually organisation-specific. It reflects the real cases, edge conditions, and regulatory requirements relevant to one particular workflow, not general model capability.
A financial services evaluation set for a customer complaints workflow, for example, might include a complaint that sounds like a simple service issue but is actually a mis-selling claim, and a complaint that was wrongly marked as resolved before. Neither of those would show up in a general model benchmark, but they’re exactly the cases that determine whether the workflow is safe to run in production.
A useful evaluation set usually includes normal cases, edge cases, past false positives and negatives, ambiguous scenarios that would need a person to judge, and known regulatory situations.
Example, across a few common AI capabilities:
| Capability | Working scenarios | Edge cases |
| Speech transcription | Clear audio, common languages, familiar terminology | Poor audio quality, accents, overlapping speech, language switching, unfamiliar vocabulary |
| Summarisation | Short, straightforward conversations | Long conversations, multiple topics, contradictory statements, ambiguous decisions |
| Alerting & Detection | Genuine policy breaches, vulnerable customer interactions, complaint scenarios | Conversations that shouldn’t trigger an alert but do, disguised language, incidents the system misses entirely |
| Compliance monitoring | Known regulatory breaches, complaints, conduct issues | Similar conversations that shouldn’t trigger an alert, new terminology, changing regulatory language |
| Human oversight | Reviewer agrees with the AI’s recommendation | Reviewer approves without really checking, or the AI is given more autonomy than intended |
Run against this set, a model upgrade or prompt change either holds steady, improves, or regresses on each row, a concrete answer to “did this change make things better” instead of a guess from spot checks.
What evaluation sets actually catch
Run consistently, an evaluation set tends to surface the same handful of failure modes:
- Prompt regressions, where a change fixes one case but quietly breaks another
- Model upgrade drift, where a new model version handles familiar cases differently
- Policy drift, where a workflow’s behaviour shifts gradually as small changes accumulate
- Recurring false positives or false negatives, especially ones already identified once before
- Inconsistent classification, where near-identical inputs get different outputs on different runs
None of these are unusual or a sign of a badly built system. They’re what happens to any AI workflow left untested over time. The value of an evaluation set is catching them before a customer, auditor, or regulator does.
Why testing once isn’t enough
Backups get tested regularly. Disaster recovery plans get drilled. AI systems can reasonably be treated the same way, because they change over time as models are updated, prompts evolve, or the surrounding workflow is modified. Continuous AI evaluation means running the same evaluation set on a schedule and after every meaningful change, turning “it seems fine” into something measurable.
The other pieces of assurance
Evaluation sets are the core of the process, but they work alongside a few other habits:
- Traceability: every outcome should trace back to what was analysed, which model produced it, and when.
- Human oversight: recording who reviewed an output, and whether they accepted, changed, or overrode it, builds a full decision history rather than a log of AI outputs in isolation.
- Audit evidence: test results, version history, and approvals need to be kept as part of normal operations, not reconstructed after the fact.
How much assurance does a workflow need
| Factor | Lower need | Higher need |
| Purpose | Search, transcription, notes | Investigations, compliance, regulatory reporting |
| Outcome | Productivity | Decisions affecting customers or regulatory obligations |
| Influence of AI | Information for a person to weigh | Recommendations routinely acted on directly |
This framework isn’t intended to classify AI systems. Instead, it helps identify workflows where stronger governance, explainability and traceability are likely to provide the greatest value.
Where the EU AI Act fits in
The EU AI Act 2 puts that same risk-based logic into law. Systems classed as high-risk, for example AI used in recruitment, credit scoring, or law enforcement, face specific obligations around testing, technical documentation, and human oversight before they can go live.
Most AI workflows in day-to-day business use, like transcription or meeting notes, fall outside that high-risk category. This isn’t legal advice, and whether a given obligation applies depends on the specific system and use case, but it’s another sign that the direction of travel is the same: know how AI is used, test it, and keep records.
Key takeaways
Picking a good model is the easy part. The harder work is showing, with evidence rather than assumption, that a workflow keeps performing as models, prompts, and the business around it keep changing. Evaluation sets, run consistently and kept alongside workflow history and review records, are one of the most effective ways to generate that evidence. That’s what Argus Archive is built to support.
Frequently asked questions
What is AI assurance?
AI assurance is the process of measuring, evaluating, and documenting whether an AI system is trustworthy and behaves as intended. It provides evidence that an AI workflow is reliable, tested, and appropriately governed, rather than relying on assumptions about its performance.
How is AI Assurance different from AI governance?
AI governance defines how AI should be used through policies, roles, and organisational controls. UK government’s Department for Science, Innovation and Technology 1 describes governance as the laws, regulations, policies, institutions, and norms used to make decisions about AI, aimed at maximising its benefits while mitigating risks and harms. AI assurance is the other half of that: the testing, monitoring, documentation, and evidence that demonstrate those policies are being followed and that AI systems continue to operate as intended.
What is an evaluation set?
An evaluation set is a fixed, repeatable collection of test scenarios used to measure AI performance consistently over time. Unlike general benchmarks, evaluation sets are built around an organisation’s own workflows, edge cases, historical failures, and regulatory requirements, making them useful for validating real-world AI behaviour.
How is an evaluation set different from a benchmark?
A benchmark measures the general capability of an AI model using standard public tests. An evaluation set measures how well a specific AI workflow performs against the scenarios that matter to your organisation, including edge cases, historical false positives and negatives, and regulatory situations.
How does Argus Archive support AI assurance?
Argus Archive AI Assurance keeps evaluation set results, workflow and prompt version history, model configuration, human approvals and overrides, and a traceable link back to the original source data, together as evidence of how an AI-assisted decision was produced and reviewed.
References
- GOV.UK Introduction to AI assurance: https://www.gov.uk/government/publications/introduction-to-ai-assurance/introduction-to-ai-assurance ↩︎
- European Commission EU AI Act:
https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai ↩︎ - NIST Artificial Intelligence Risk Management Framework:
https://www.nist.gov/itl/ai-risk-management-framework ↩︎ - ISO/IEC 42001:2023
Information technology – Artificial intelligence – Management system:
https://www.iso.org/standard/42001 ↩︎ - OECD AI Principles:
https://oecd.ai/en/ai-principles ↩︎ - European Commission EU AI Act:
https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai ↩︎ - GOV.UK Introduction to AI assurance: https://www.gov.uk/government/publications/introduction-to-ai-assurance/introduction-to-ai-assurance ↩︎
Disclaimer
This article is provided for informational purposes only and does not constitute legal advice.
Organizations should consult their data protection officer or legal counsel for guidance on their specific situation and applicable national requirements.
