Best practice 9

State human-in-the-loop requirements explicitly

Practicepro·con

• Every agentAgentA system built around a language model that carries out multi-step work: it plans, calls tools, reads the results, and decides what to do next. The model supplies the reasoning; the surrounding harness does the acting. Distinct from a chat model that answers one prompt at a time. use case should state its human-in-the-loopHuman in the loop (HITL)A human checkpoint in an agentic workflow. Each practice states what oversight it requires: mandatory, optional, during the process, or as a final check. (HITL) level plainly.3·0
  For

  • ELIXIR TF Agentic AI: agenda and rolling best practice (2026) Best Practice item 7
  • EU AI Act (Regulation (EU) 2024/1689) Article 14
  • NIST AI Risk Management Framework (2023, 2024) AI 600-1: oversight calibrated to risk
  AgainstNone recorded
• HITL involvement can be mandatory or optional; it can range between tight in-process control and a mere final check.
• A workflow stating no HITL involvement means it has full autonomy.
• Left unstated, no one knows where a person is required and where the agent acts alone.
• Stating the level is necessary, but not sufficient: the check has to be designed so it actually works in deployment.4·0
  For

  • Moral Crumple Zones (2019) moral crumple zones: token oversight makes the human a scapegoat
  • Flaws of Policies Requiring Human Oversight (2022) human-oversight mandates often fail unless designed for effectiveness
  • EU AI Act (Regulation (EU) 2024/1689) Article 14 requires oversight to be effective; names automation bias
  • A safer framework for patient data in AI-for-Science grants (2026) agent-run code can transmit data without a deliberate upload
  AgainstNone recorded
• HITL engagement provenance should be recorded according to BP07.

For practitionersFor providersFor governance

Know the HITL level of the agent workflows you use, and do not assume a check that has not been stated. For anything that changes results or affects others, keep a real human decision point even if the tool lets you skip it. Do not automate approval; acceptance without scrutiny is not oversight.

State the HITL level for each use case you support, and build the check into the workflow; do not count on the user to remember or even know. Match the check to the stakes, have the system enforce the stated level, and design the check so the human can actually understand and override the output. Try to facilitate understanding and to prevent "just clicking approve."

Stated HITL levels make oversight auditable. Require each use case to declare its level and enforce it. Make stronger checks mandatory where actions have real consequences. Test that the oversight works in practice rather than assuming a named reviewer is enough. This includes peer-review integrity: confidential manuscripts should not be uploaded to external models, and reviewer AI use should be declared.

Reasons

How much oversight is right depends on the use case. A low-stakes lookup and an action that changes a record or a result do not need the same check. If the level is unstated, oversight is left to chance, which is neither reliable nor checkable. But a stated level is not a guarantee. There is strong evidence that human oversight often fails in practice: people defer to plausible-looking output (automation bias), rubber-stamp at scale, and can be left holding blame for systems they could not realistically control (for instance, due to volume). So the check must be designed for effectiveness, giving the human what they need to understand and override. Pipelines need to be assessed for realistically achievable volumes. The four labels used here (mandatory / optional, in-process / final check) are a useful shorthand, not settled standards. Agent autonomy is a spectrum from tight supervision to full autonomy. Stating and enforcing HITL levels fulfils part of BP04 at the single use case level.

Examples

• A workflow never states its HITL level; each user assumes someone else checks, and an unreviewed agent edit reaches a shared record.
• Every use case instead declares its level from a short list (mandatory, optional, in-process, final check), and a workflow that declares none is understood to run at full autonomy, so the choice is explicit.
• A pipeline has a mandatory "human approves each item" step, but at hundreds of items an hour the reviewer clicks approve without reading; the check exists on paper and fails in practice.
• The check is designed so the human can actually judge: it shows what changed, at a realistic volume, with approve, edit, veto, and stop, and the strength of the check is set by the stakes.
• The stated level is enforced by the system rather than left to habit, which ties it to the guardrails in BP04, and the HITL engagement is recorded in provenance (BP07) so oversight is auditable instead of assumed.
• A reviewer uploads a confidential manuscript to an external model against the stated boundary of the conference for a high-stakes task. They submit the AI-generated review without declaring the AI use. The conference had inserted a prompt injectionPrompt injectionAn attack that hides instructions inside content an agent reads (a web page, a document, a tool description), so the model follows them as if they came from the user. The user never sees the injected text, which is what makes it dangerous. Tool poisoning is one form, where the hidden instructions sit in an interface's tool description. into their review version and catches the reviewer with the prompted output form. The reviewer is barred from conference participation for two years.

Sources

• ELIXIR TF Agentic AI: agenda and rolling best practice (2026), Best Practice item 7.
• Agentic AI in the higher-education system (2026), §4.1 (explicit intervention points) and §6 (defined escalation and control). Consistent with, grounding only.
• EU AI Act, Article 14 and NIST Generative AI Profile (AI 600-1). Oversight must be effective and calibrated to risk; Article 14 names automation bias as a hazard the overseer must be able to counter.
• Elish, Moral Crumple Zones (2019) and Green, Flaws of Policies Requiring Human Oversight (2022). Evidence that human oversight frequently fails in practice; a stated level must be designed for effectiveness.

Change history

• 2026-07-27: Renumbered from BP08 to BP09 on inserting the new BP01 (match the method to the task).
• 2026-07-27: Rewrote Examples as concrete scenarios (actor, action, outcome), including anti-patterns (unstated level, rubber-stamping at volume), replacing restatements of the practice.
• 2026-07-26: Renumbered from BP08 to BP09 in the reordering. Added grounding (EU AI Act Article 14, NIST AI 600-1) and a caveat that a stated level is necessary but not sufficient (automation bias, rubber-stamping, moral crumple zones; Elish 2019, Green 2022), plus a peer-review confidentiality instance.
• 2026-07-25: Created from the ELIXIR TF distillation (hook 8), grounded in the HFD paper.

Discussion