AI ethics

Give People a Way to Challenge an AI-Assisted Decision

Design and rehearse an appeal path that connects an affected person with evidence, a reviewer who can act, and a reasoned answer in time to matter.

Top One Percent AI · Practical guidance with linked sources.

A person can be told that AI influenced a decision and still have no practical way to change it. They may know which system was used, yet be unable to correct a mistaken record, reach someone with authority, or learn whether their objection was considered. An ethics session becomes more useful when it follows that person's next steps instead of ending with disclosure.

Consider a fictional organization that uses AI to help prioritize training applications. An applicant is ranked poorly because an old record says they lack a qualification. A functioning challenge process must let them submit the correction and have the decision reconsidered while a training place can still be offered.

Define what someone can challenge

The OECD AI Principle on transparency and explainability calls for information that enables adversely affected people to challenge an AI system's output. NIST's AI Risk Management Framework includes appeal and override mechanisms within post-deployment monitoring under MANAGE 4.1. These provide a foundation for process design, not a universal statement of legal rights. Applicable duties depend on the jurisdiction and use case.

A practical process should accommodate several kinds of challenge. The underlying data may be wrong. Relevant evidence may have been missed. A policy may have been applied incorrectly. The policy itself may deserve scrutiny. These are different problems and may need different reviewers.

Do not require the person to diagnose a model failure before asking for help. “This qualification is missing from my record” should be enough to start the appropriate review. The organization can determine later whether the problem originated in data collection, retrieval, inference, or human judgment.

Make the route usable

For an organizer workshop, ask teams to design the path from the decision notice to an actual review. This is a recommended exercise, not a prescribed regulatory template.

The notice should identify where to ask for reconsideration, what information would help, and what happens next. Explain whether any deadline or consequence continues during the review. Choose response targets that the organization can support, and provide an escalation route for urgent cases. Avoid promising an outcome before the evidence has been considered.

Test the route with someone unfamiliar with the organization. Can they find it without knowing the name of the AI system? Can they use it without repeating information already provided? Is there an accessible alternative to a complex online form? A channel that technically exists but cannot be navigated is weak evidence of a working remedy.

Give the reviewer evidence and authority

A reviewer needs the decision record, the relevant policy, the information used, and the person's correction or objection. Include the AI contribution where it can be identified, as well as any subsequent human changes. Restrict access to the people who need it and avoid collecting unrelated personal information merely because a dispute exists.

The review should answer the objection, not simply run the same request through the same system again. If a qualification was omitted, verify that qualification and reassess the case under the applicable policy. If the objection concerns the policy, send it to someone able to examine that issue rather than closing it as a data error.

Specify what the reviewer can do. Can they correct a record, reverse a decision, pause its effect, or require another assessment? Where they cannot act, identify who can. For consequential cases, consider separating the review from the original decision maker and checking for conflicts of interest. These are governance recommendations to adapt to the setting.

Close the case and examine the pattern

Communicate the review outcome in ordinary language. State what was considered, what changed or remained unchanged, and why. If further review is available, explain how to request it. Where a record was corrected, confirm whether the correction reached the systems that may reuse it.

Reviewing individual cases can also reveal recurring defects. Track categories of objections, review delays, corrections, and unresolved barriers. Interpret these measures together. Few appeals could reflect accurate decisions, but could also reflect a hard-to-find process. A low reversal rate alone does not establish that the system or review is fair.

Run a challenge-path rehearsal

Give three participants the roles of affected person, reviewer, and process observer. Provide a fictional decision notice, a short policy, and an incomplete decision record. Let the affected person introduce a correction that could change the outcome.

Ask the observer to record every point where progress stops. Missing evidence, unclear authority, inaccessible channels, and unanswered deadlines should become named improvements with owners. End by asking whether the person received a reasoned answer in time to matter. That question gives an ethics session a concrete outcome beyond agreement that human oversight is important.