Engineering Trust: Assurance When the Human Leaves the Point of Work

For most of my career, assurance has rested on a simple assumption: someone is responsible for the thing being controlled. Someone holds the account, exercises judgement over what they see, and can explain what happened if asked. When something goes wrong, that same person can be held accountable for it. Replace the person with an AI agent, an autonomous system or an uncrewed platform, and that assumption becomes much harder to sustain. The question is one I recognise from naval logistics: what happens to assurance when there is no longer a person standing at the point of work?

This territory was first explored in a presentation I gave to the Global Skill Development Council on 19 February 2026, mapping naval logistics assurance practice against ISO/IEC 42001, the international standard for AI management systems. Recent work from Deloitte and Boston Consulting Group on AI-enabled assurance and agentic systems suggests that both fields are confronting the same problem from opposite directions (McNulty et al., 2026; Coppola and Kleppe, 2026). The problem is how to maintain trust and accountability when authority, activity and decision-making move away from the person who has traditionally stood at the point of work. The industries differ widely, yet the problem is shared.

The point of work matters

Assurance can be generated at source, built into how a process operates, as close as possible to where the work takes place (Deloitte, 2026). The principle is already familiar in naval logistics. Routine checks on stores and equipment accounts fall to the most junior people in the department, because carrying out the check is how they learn to own the risk. The officer responsible for the department then applies judgement on top: does what they are seeing make sense, and does the account reflect what is happening on the ship?

When the answer is yes, a short entry confirming that nothing significant was found can be a complete record. Its value comes from the competence and judgement behind it. A short entry from someone close to the risk, who can explain their decision if challenged, may carry more weight than a long checklist. The same logic could apply in AI governance, where the quality of a control matters more than the volume of paperwork around it.

When inspection needs to mean something

In naval logistics an independent inspection can produce a graded outcome. A department that fails outright has to rectify the problem and be re-inspected before anything proceeds. The result carries consequences, and the contrast with AI governance is instructive. If an AI system fails a material safety, fairness or control test and the assurance function can only document the finding, the mechanism is limited to reporting the problem. I would expect a stronger model to give assurance enough authority to change the operational outcome.

Naval inspection regimes are organised around an identifiable unit to inspect: a department, a site or an account with a defined boundary and somebody answerable for it. Activity below that boundary is hard to see. Agentic AI makes this acute, because an agent can operate across applications, accounts and workflows without fitting inside any unit an existing inspection regime recognises. The inspection model may still be sound while the boundary drawn around it no longer matches where the activity happens.

The handover question

ISO/IEC 42001 requires a named accountable owner for every AI system and an agreed backup and escalation route, according to summaries of its provisions on AI roles and responsibilities. Those summaries leave open whether the system's risk position should be reassessed at the point accountability transfers. Naval logistics answers that directly. When responsibility for an account passes to a new officer, the incoming custodian can be required to accept it only after it has been audited and any findings recorded. The principle travels well beyond the military procedure, since a change in accountability offers a natural trigger to reassess risk.

In agentic banking, the question sharpens when an AI agent acts under authority originally granted to a human (Coppola and Kleppe, 2026). An agent may inherit access granted to a supervised human, while the resulting audit trail identifies the credential without identifying the agent, the delegation or the rationale behind the action. The handover question then becomes who authorised this specific action, under what delegation, and what evidence shows how that authority was exercised.

The assumption underneath

The junior rating learns to own the risk because they are physically present with the materiel. The handover audit works because accountability moves from one named custodian to another, and the short record is meaningful because a competent person looked and can explain what they saw. Each mechanism above therefore depends on a person standing at the point of work, and removing the person removes the thing that made each one work.

Research into autonomous military systems reaches a similar point (Schipper and Lagendijk, 2026). The researchers interviewed eleven Royal Netherlands Army stakeholders about how accountability is preserved once decisions are delegated to autonomous weapon systems. They found responsibility distributed across developers, procurement organisations, commanders, operators and the system itself. No single actor retained full situational control at the decisive moment, a condition they term an accountability vacuum. One interviewee put it plainly: "If you're going to hold somebody accountable, there has to be a level of predictability in that it was going to go wrong." The details differ from banking and logistics, and the pattern suggests that responsibility can become distributed faster than accountability mechanisms can follow it.

When the human is no longer holding the asset

Materiel held or moved by an uncrewed or automated system may operate under authority delegated to a person who is nowhere near it. If the asset moves under pre-authorised instructions and is lost, there may be a named account, a system log and even telemetry, but no human custodian who was present at the time. An investigation would then need to establish what the system was authorised to do and what it did. It would also need to establish which agent or process executed the action, what authority was delegated, and what information and evidence explain the decision.

These are old assurance questions, and autonomous systems are making them urgent again. The vocabulary of custody, built for stores accounts and equipment logs, may give AI governance a way of thinking about accountability that does not depend on the person who used to be standing there. What this means for physical supply chains, where telemetry can become the only witness to a loss, is traced in The Convergence of Cyber-GRC and Physical Logistics.

Deciding what is worth keeping

Naval logistics assurance separates checks that are legally or operationally non-negotiable from discretionary ones that are reviewed according to their risk. AI-enabled assurance is moving towards embedded and preventative controls, and the naval distinction suggests choosing which controls to automate and how much documentation each event warrants. For AI governance, that would mean identifying the load-bearing controls, those protecting safety, legal exposure, material bias thresholds, security and availability, and concentrating effort there.

Provision 29 of the UK Corporate Governance Code, in force for financial years from January 2026, requires boards to declare whether their material internal controls are effective. Continuous controls monitoring platforms already check critical items hourly or daily (Vanta; Sensiba LLP, 2026). A related point comes from inside banking, where validation built for periodic, human-paced review does not hold once a system is composing decisions continuously (Coppola and Kleppe, 2026). The principle naval logistics shares with that shift is risk-based frequency: how often a control runs is set by the consequence of failure and the speed at which the risk can change, whether the control is automated or manual.

Behind the credential

Beside naval practice, the AI governance literature is working through the same four concerns in a new setting. They are who owns a system, which controls bear load, what the record shows when authority is inherited, and whether activity has slipped below the level that inspection recognises. Naval assurance answered the first by attaching every account to a named custodian and treating a change of custodian as a reason to look again. The third may be the hardest to carry across. An AI agent acting under a delegated credential can leave a record that identifies the credential and little else. A record of that kind would not survive the handover audit described above. My own view is that the named owner comes first. Once each AI system has one, and a change of owner prompts a fresh look at its risk position, the other three concerns have someone accountable for answering them.

Where this argument goes next

Each assurance mechanism loses the thing that made it work once the person leaves the point of work. Military assurance offers no ready-made model for AI governance, because the environments and the consequences of failure differ too much. It does expose an assumption that otherwise stays hidden: traditional assurance was built around identifiable custody, where someone owns the asset, controls the process, exercises judgement and can be asked what happened. AI and autonomous systems are separating those things. The person who owns the system may not have initiated the action, and the person who authorised it may not have observed the outcome. The next task for assurance may be establishing whether authority, action, evidence and accountability still connect to each other at all.

The Royal Navy's pivot towards a Hybrid Navy, a mixed force of crewed and uncrewed platforms, raises that question directly, with Atlantic Bastion and Project BEEHIVE among the publicly named initiatives behind it. In my own assessment, the publicly available material has not yet set out how custody is verified, who is accountable if an asset is lost or damaged, or what evidential standard the record must meet once materiel is held by an uncrewed system. That question will be the subject of my dissertation for the Defence Logistics Staff Course, drawing on eighteen years across movements and logistics. Both disciplines are built around a person at the point of work, and both are now asked to answer for a force that is deliberately removing the person from some of those points. The question raised at the start of this essay is one of engineering trust once the human point of custody is no longer there to vouch for it. Artificial intelligence is one of several places where it now arises.

The views expressed here are my own and are based on general observations about logistics assurance and AI governance. They draw only on published sources and do not represent the position of the Royal Navy, the Ministry of Defence or any organisation with which I work. Naval practice is described in general terms. Programmes named here, Hybrid Navy, Atlantic Bastion and Project BEEHIVE, are identified by name only, drawn from material already in the public domain; nothing internal to the Royal Navy or the Ministry of Defence is reproduced, and no platform, operation or capability detail is described.