From December 2027, organisations using ‘high‑risk’ AI — systems deployed in areas where mistakes carry serious consequences — will need to demonstrate that human oversight was informed, effective and embedded in everyday decision‑making. Many may feel they have until the deadline to comply, but in practice they have until the deadline to show that capability already exists. (More information on the deadlines is available here.)

This is where many organisations are unprepared. ‘Human in the loop’ has long been treated as a safeguard in this area, but the Act requires more than proving a person was present. Regulators will want to understand how human judgement was applied, why decisions were trusted, and whether oversight genuinely shaped the outcome.

The Limits of ‘Human in the Loop’

If a person is involved, the assumption may be that oversight exists. But a human can be present without having the context, evidence or understanding needed to make a meaningful decision. The issue is not the AI system itself, but whether the organisation can provide and preserve enough understanding for judgement to occur. 

When organisations are asked to demonstrate oversight, they often produce audit trails, workflow records and governance reports that show a human was there. What these records rarely show is how that human reasoned through the decision. The EU AI Act raises the bar by asking organisations to demonstrate the effectiveness of the human oversight process.

Agentic AI Makes Oversight Harder to Prove

This challenge becomes greater as agentic systems evolve. Agentic AI systems can initiate actions, adapt their behaviour and make decisions based on changing conditions. 

When AI behaves in this way, an organisation cannot rely on checking outputs or validating isolated decisions as proof of oversight. They must be able to explain why the system’s actions were trusted, what information shaped human judgement, and how oversight was exercised when the system acted autonomously. The traditional idea of ‘human in the loop’ becomes harder to defend if the organisation cannot show how a human actually understood and influenced the decision.

The Missing Layer: Understanding, Not Just Records

What is missing is not another governance process, but a way of preserving organisational understanding. Organisational workflows and processes are often not structured to capture the reasoning behind decisions. They record what happened, who did it, and when it occurred. But they rarely preserve the evidence that was reviewed, the assumptions that were made, or the interpretations that shaped the final judgement.

This missing layer is exactly what regulators will look for when assessing whether oversight was genuine. Without it, organisations cannot convincingly demonstrate compliance or trustworthiness. A human may have been ‘in the loop’, but the organisation cannot show how that human actually exercised judgement.

Why Decision Context Matters

As AI systems become more capable, the burden shifts from proving that humans were involved to proving that humans were effective. Organisations must be able to show how decisions were made, what evidence was considered, and how risks were interpreted at the time.

If the context is not preserved, organisations cannot explain their own judgement, and therefore cannot demonstrate that their oversight met the required standard for high‑risk AI.

Building Organisational Memory for AI Governance

The ability to capture this evidence and context is the foundation of organisational memory. Organisations that invest in preserving judgement today will be better prepared for regulatory expectations, internal assurance and the safe deployment of more autonomous systems. Those that continue relying on procedural oversight will find themselves exposed when the EU AI Act’s requirements take effect.

The EU AI Act is prompting organisations to rethink what ‘human in the loop’ really means. Their response will define which organisations can confidently use advanced AI, and which will struggle to show how human judgement shaped their decisions. 

How VeriNote Can Help

VeriNote gives organisations a practical way to capture the evidence they considered, the interpretations they made, and the context in which decisions took place. Rather than depending on incomplete records or trying to piece decisions together after the fact, organisations can retain a clear account of how human understanding shaped the outcome.

By preserving the evidence, interpretations, assumptions and context that informed human judgement, VeriNote allows organisations to show not only what decisions were made, but how and why those decisions were trusted. It keeps this reasoning intact and provides a connected, durable record of organisational understanding. This supports governance, strengthens accountability, and gives organisations the assurance they need as AI systems take on more autonomous roles.