Skip to content
ContestabilityA research
publication
HomeResearch / NIC program

A research program on meaningful challenge

What information makes an institutional determination challengeable—and when does an institution owe a person a consequential response?

Author
Kerry Thornhill
Status
Research in development
Source date
Web version
2026-10-23 · v3

Kerry Thornhill

MS in Interdisciplinary Studies · In progress · University of North Texas

Anuradha and Vikas Sinha College of Artificial Intelligence and Advanced Analytics

Two companion documents define this research program: an argument map that sets out the shared conceptual core of Paper A (now available as a preprint) and Paper B, and a 36-month research plan that tests it. Both are working documents; the plan is proposed research, and this site reports no completed study results. A separate philosophical inquiry, What Counts as a Reason: Institutional Formation and the Limits of Rational Reconstruction, examines questions that inform NIC and Paper B.

The argument map

Argument map · v. 5 · 12 October 2026 · One argument map — three uses: Paper A | Paper B | research plan

An algorithmic score, classification, prediction, recommendation, rank, flag, or diagnosis becomes an institutionally operative determination when institutional adoption or material reliance makes it structure treatment, opportunity, visibility, priority, burden, or presumptive status. Paper A asks what informational functions meaningful contestability requires where contestability is independently required or promised; Paper B separately asks when a private institution’s evaluative use itself generates a pro tanto duty to provide that contestability.

  1. 1Institutionally operative determinationIdaho adopted a statistical budget tool’s output as each participant’s Medicaid waiver budget.
  2. 2Plausible material objectionA participant whose budget fell could reasonably object that the tool used wrong or incomplete records, that its inference was unreliable, or that the review standard was undefined.
  3. 3Epistemic task needed to investigate itAssessing that objection requires the data the tool used, the tool’s testing record, and the standard applied — none of which the 2011 notices supplied.
K.W. v. Armstrong, 789 F.3d 962, 971–74 (9th Cir. 2015); 180 F. Supp. 3d 703, 717 (D. Idaho 2016)

Mentorship and review

Dr. Shomik Chakrabarti, Clinical Assistant Professor, University of North Texas

Dr. Shomik Chakrabarti (University of North Texas) provides methods mentorship and critical review of the NIC framework; his challenge to its six-function classification informs a planned comparison to be preregistered before testing.

Six candidate informational functions

  1. 01

    Determination and consequence

    Task Identify what the institution concluded and what practical effect followed.

    Defect it exposes The output is named but the operative decision or its consequence is obscured.

  2. 02

    Sociotechnical roles and evidence control

    Task Identify who generated, supplied, interpreted, adopted, modified, or controls the grounds, including automation’s actual contribution and any human intervention.

    Defect it exposes A human label conceals actual automation, intervention, or vendor control.

  3. 03

    Data, source, and quality

    Task Assess provenance, accuracy, currency, linkage, omission, proxy use, and constitution of the evidence base.

    Defect it exposes Wrong, stale, incomplete, mislinked, or unrepresentative data shaped the result.

  4. 04

    Governing-standard–operation fit

    Task Compare the authorized standard with the target, construct, proxy, rule, or objective implemented.

    Defect it exposes The system accurately computes an unauthorized or wrong criterion.

  5. 05

    Evidential warrant, uncertainty, and threshold

    Task Assess whether the evidence supports the inference and whether reliance at the selected threshold is warranted.

    Defect it exposes Weak validation, subgroup error, uncertainty, or threshold proximity undermines reliance.

  6. 06

    Contextual applicability

    Task Assess whether the system, evidence, and assumptions transport to the present person, population, place, time, workflow, and exceptional circumstances.

    Defect it exposes Domain shift, atypical circumstances, or omitted context defeats application.

Institutional uptake

Institutional uptake is external to the register. It is not a seventh informational function. It is the institutional relation (Paper B: answerability) that connects an informed objection to a competent and sufficiently independent forum capable of reasoned response and consequence — correction, modification, suspension, systemic change, or warranted reaffirmation.

Two tracks

Functional track — Paper A

Let d be an independently specified material objection within scope; Od its assessment task; oi a candidate operation within function Fi; RC the bounded feasible-route set under conditions C. oi, or a faithful equivalent, is conditionally necessary when the regime purports to permit d, assessing d requires Od, some route in RC performing oi reaches the prespecified adequacy floor on Od, and no route in RC that omits oi and its faithful equivalents does. Necessity attaches to the operation and its task; the sixfold partition remains revisable.

Duty track — Paper B extension

Functional necessity does not entail institutional duty. Paper B proposes a pro tanto duty of Relational Institutional Answerability where four conditions converge: the institution intentionally makes a purportedly reasons-responsive judgment operative; that use foreseeably matters to the person, individually or cumulatively; the institution or its accountable decision chain holds exclusive or materially superior control of the grounds needed to investigate a reasonable objection; and the affected person lacks a reasonably adequate countervailing route (avoidance, verification, correction, representation). The paper must defend why that relationship gives the affected person a claim to answerability and why a proportionate challenge route is an appropriate constraint. The proposed argument connects operative evaluative use with responsibility for epistemic and correction risks, examines control over judgment and challenge as a possible form of arbitrary private power, and considers the least restrictive adequate response. Read the developing argument →

The sense of “necessary”

Not legally required everywhere, not personally possessed by every affected party, not jointly sufficient for justice, not causally indispensable in every case, not proven exhaustive. A contestation may activate only some functions. Epistemic fidelity determines faithful equivalence; timeliness, independence, accessibility, and usability determine delivery adequacy; institutional uptake remains external. The partition is merged, split, expanded, or domain-limited when evidence defeats it; two undetermined nulls across prespecified settings downgrade the bounded necessity claim.

Three claims

  1. 1

    Objection–operation claim

    Meaningful contestation requires the capacity to investigate material objections, not merely receipt of an explanation, complaint form, or technical dossier.

  2. 2

    Staged-availability claim

    Required tasks may be enabled through baseline, preservation, activated access, protected mediation, and usable support; necessary availability does not mean universal immediate disclosure.

  3. 3

    No-substitute claim

    An operation is necessary only relative to an independently specified objection, defined conditions, and a bounded feasible-route set in which no route that omits the operation and its faithful equivalents reaches the adequacy floor on the same task; a faithful equivalent changes delivery, not necessity.

Research hypothesis

Research hypothesis

Test whether the register covers naturalistic objections and survives a preregistered rival partition (the merger of the third, fourth, and fifth functions); whether its operations are conditionally necessary rather than merely helpful, omission falling below a prespecified adequacy floor while a faithful substitute clears it; whether reliance on labeled advice stays contributory, valid defeaters reducing it while warranted reliance is retained, or becomes preemptive; separately, whether judged warrant and chosen disposition diverge under institutional constraint; and whether investigative capability survives held-out transfer.

The 36-month research plan

36-month research plan · v. 6 · 12 October 2026 · Testing Informational Conditions for Meaningful AI Contestability

Research question

Which informational functions enable people to investigate AI-supported decisions while preserving warranted reliance, and how can those functions become a usable product across disciplines?

This 36-month project will complete the initial two-function study in year one, test the remaining four functions during years two and three, and deliver a working contestability toolkit with evaluated transfer across disciplines. Findings will determine which functions remain distinct and how the product changes.

Three phases, 36 months

  1. 1Phase 1 · Months 1–12

    Complete the Initial Two-Function Study

    Functions
    3 · Data, source, and quality
    4 · Governing-standard–operation fit (one operation: proxy-to-criterion fit)
    Work
    Corpus, pilots, preregistration · two experiments
    Revision point
    Month 12
    Deliverables
    First working toolkit modules
  2. 2Phase 2 · Months 13–24

    Test Two Further Functions

    Functions
    1 · Determination and consequence
    2 · Sociotechnical roles and evidence control
    Work
    Public-benefits cases · belief/action and staged-access tests · transfer recruitment · replication slot
    Revision point
    Month 24
    Deliverables
    Both completed tests, the staged-access comparison, and an integrated inquiry and review prototype
  3. 3Phase 3 · Months 25–36

    Test the Final Two Functions and Transfer

    Functions
    5 · Evidential warrant, uncertainty, and threshold
    6 · Contextual applicability
    Work
    Health-care cases (25–30) · field uptake studies where access is secured (25–36) · held-out transfer and replication slot (31–36)
    Revision point
    Month 30
    Deliverables
    Toolkit released at 36

At months 12, 24, and 30, the plan retains, merges, splits, adds to, or restricts the scope of functions against the evidence, and preregisters changes. Null results will be reported. Studies will not affect live decisions.

The month-36 toolkit

Month 36 · Release

By month 36, I will release a working contestability toolkit for researchers and process designers.

Users will configure and evaluate the complete process from notice through inquiry, evidence, reasoned response, and recorded action, including cases with unavailable evidence or limited reviewer authority.

  • Connected affected-party and reviewer interfaces
  • A case generator
  • A record linking grounds to evidence and disposition
  • Tested modules
  • Scoring tools
  • An adaptation guide

Documents and versions

Kerry Thornhill · Anuradha and Vikas Sinha College of Artificial Intelligence and Advanced Analytics · Interdisciplinary Studies · University of North Texas. Core anchors: K.W. v. Armstrong, 789 F.3d 962, 971–74 (9th Cir. 2015); 180 F. Supp. 3d 703, 717 (D. Idaho 2016), official slip op. at 11–15, 22–25, 36 · Hardwig 1985, 335–37 · Lange 2026, 180–81 · duty analogues: Anderson 1999, Bovens 2007, Dagan & Dorfman 2016, Pettit 2012 — adjacent support, not independent derivations.

Four connected inquiries

Paper A develops the informational account; Paper B investigates the proposed duty of answerability; the empirical program tests selected claims; and What Counts as a Reason: Institutional Formation and the Limits of Rational Reconstruction examines the philosophical questions of institutional formation and rational reconstruction. Each has its own question and burden of support.

Paper A is available as a preprint. The philosophical paper is in preparation with a preprint planned for December 2026; Paper B is in preparation with a preprint planned for January 2027. The studies remain proposed.

Selected intellectual context

The program builds on established work on contestability and reviewability. These are entry points to the surrounding literature.