Human-in-Charge AI Review

Use this review before approving one AI use case in home or
community care.

For training and workflow-design purposes, use fictional or
appropriately protected information. Do not enter names, health details
or other sensitive personal information on the webpage.

1. Name the use case

  • What bounded problem are we trying to solve?
  • Who will use the output?
  • Who could be affected if the output is wrong or unavailable?
  • What does success look like without promising a clinical
    outcome?

2. Classify the role

Assist

  • The task is limited and reversible.
  • A person can readily verify the result.
  • The output does not determine care, rights, eligibility or safety
    action.

Review

  • The output may influence care, prioritization, staffing, monitoring,
    documentation or communication.
  • A qualified person has time, evidence and authority to
    disagree.
  • Escalation and override are built into the workflow.

Stop

Do not delegate final authority for:

  • emergencies;
  • diagnosis or treatment;
  • medication changes;
  • consent or information-sharing decisions;
  • care-plan approval;
  • service restriction, eligibility, discipline or employment action;
    or
  • serious or life-changing communication.

3. Define the data boundary

  • What information may enter the system?
  • What information is prohibited?
  • Is identifiable information necessary?
  • Is the data retained or used for model improvement?
  • Where is it stored and processed?
  • Who may access it?
  • How is it deleted?
  • Which contracts, laws, policies and permissions apply?

4. Require useful evidence

  • Intended purpose is documented.
  • Relevant populations, settings and languages are described.
  • Performance measures and limitations are available.
  • Fairness or subgroup evaluation is available when relevant.
  • Updates and model changes are communicated.
  • Medical-device status and intended use are verified when
    applicable.

5. Design human review

Requirement Decision
Named reviewer
What the reviewer must check
Source information available
Conditions requiring escalation
How disagreement is recorded
Who makes the final decision
How the affected person reaches a human

6. Test failure conditions

  • Missing information
  • Conflicting information
  • False alert
  • Missed alert
  • Different language or communication need
  • Reviewer unavailable
  • System outage
  • Person declines the recommendation
  • Request for correction or complaint

7. Approve with an exit plan

  • Owner for the use case is named.
  • Monitoring measures are defined.
  • Review date is scheduled.
  • A pause trigger is defined.
  • A safe manual workflow exists.
  • The organization can stop the use without losing essential care
    information.

Final decision

  • Approve as Assist
  • Approve with required Human Review
  • Revise and retest
  • Stop—do not delegate this use case to AI

Important: This tool provides general educational
and operational information. It is not medical, legal, privacy,
compliance, procurement or emergency advice. Requirements depend on the
tool, intended use, organization and jurisdiction.