Governing AI-based referral recommendations means writing down what the model is for, which inputs it may use, how a recommendation is explained, who can override it, how fairness and drift are monitored, what is logged, and who reviews the results on a schedule. Treat referral matching as an operational decision system under continuous oversight — not a feature that was approved once at purchase.
Quick answer
- Define purpose and prohibited uses before deployment.
- Enumerate permitted inputs; treat everything else as prohibited or verify-before-use.
- Require an explanation per recommendation, not a model card alone.
- Keep human override one click away and log the reason.
- Monitor fairness, access, override patterns, and drift on a fixed cadence.
- Make vendor model changes a controlled, notified event.
This is operational governance guidance, not legal advice. It is distinct from our patient choice and referral steerage compliance article, which addresses the choice and steerage question specifically; here the subject is the control environment around the algorithm itself.
Definition — referral matching AI: software that ranks or recommends destinations for a referral using patient, coverage, network, clinical, access, and preference data. It is assistive when a human selects from ranked options and operational when it also executes downstream steps such as outreach or authorization.
The 15-control checklist
| # | Control | What "in place" looks like |
|---|---|---|
| 1 | Documented purpose and scope | Written statement of intended use, populations, specialties, and prohibited uses |
| 2 | Permitted input register | Signed list of inputs, source system, refresh cadence, and justification per input |
| 3 | Data provenance and quality gates | Lineage for each input, freshness thresholds, and automated acceptance tests |
| 4 | Explainability per recommendation | Ranked factors and weights visible to the user at the moment of decision |
| 5 | Human override | One-click override, structured reason codes, no penalty or friction for overriding |
| 6 | Patient choice capture | Preference recorded as an input; declines honored and logged |
| 7 | Clinical appropriateness boundary | Model does not determine medical necessity or coverage denial |
| 8 | Pre-deployment validation | Retrospective evaluation on local data with documented acceptance criteria |
| 9 | Fairness and access monitoring | Segmented outcome monitoring with defined thresholds and escalation |
| 10 | Drift and stability monitoring | Input distribution and output distribution monitoring with alerting |
| 11 | Immutable audit logging | Per-recommendation record of inputs, version, output, user action, and timestamp |
| 12 | Incident response | Severity definitions, containment steps, rollback path, notification duties |
| 13 | Vendor change control | Advance notice of model changes, change log, re-validation trigger |
| 14 | Access, security, and privacy | Role-based access, minimum necessary data, encryption, retention, agreements |
| 15 | Governance reporting cadence | Named committee, fixed schedule, standing report, documented decisions |
Print it. Score each control red, amber, or green. Amber items need an owner and a date; red items need a decision about whether to deploy at all.
Governance ownership and RACI
| Activity | Clinical leadership | Compliance / privacy | Data / analytics | Security | Network operations | Vendor |
|---|---|---|---|---|---|---|
| Purpose and prohibited uses | A | R | C | I | C | I |
| Permitted input approval | C | A | R | C | C | C |
| Pre-deployment validation | A | C | R | I | C | C |
| Explainability requirements | A | C | C | I | R | R |
| Override and appeal workflow | A | R | I | I | R | C |
| Fairness and drift monitoring | C | C | A/R | I | C | C |
| Audit logging and retention | I | C | C | A/R | I | R |
| Incident response | C | A | R | R | C | R |
| Vendor change control | C | C | R | C | A | R |
| Governance reporting | A | R | R | C | C | I |
A single accountable owner per row prevents the common failure where monitoring exists but nobody reads it.
Permitted, prohibited, and verify-before-use inputs
| Input | Status | Notes |
|---|---|---|
| Referral specialty and clinical question | Permitted | Core clinical fit |
| Diagnosis and relevant clinical history | Permitted | Minimum necessary for appropriateness of destination |
| Coverage, plan, and product; network participation | Permitted | Required for in-network determination |
| Measured provider access and availability | Permitted | Access is a patient-facing outcome |
| Published quality measures and outcomes | Permitted | Document the source and vintage |
| Relative cost tier | Permitted with limits | Never presented as a directive; keep contracted rate detail out of clinician screens |
| Geography, travel time, transportation needs | Permitted | Access equity input |
| Language and documented social needs | Permitted | Used to improve match, never to restrict options |
| Patient stated preference | Permitted and prioritized | Must be capturable and honored |
| Race or ethnicity | Prohibited as a matching input | Appropriate for fairness monitoring, not for ranking |
| Payer margin or profitability of the encounter | Prohibited | Not a patient-benefit input |
| Individual clinician productivity or compensation | Prohibited | Creates a conflict of interest in routing |
| Proxy variables for protected characteristics (e.g., granular geography used alone) | Verify before use | Assess for proxy discrimination before enabling |
| Consumer or purchased behavioral data | Verify before use | Provenance, consent, and necessity review required |
Fairness monitoring may use demographic data even where matching may not. Keep the two uses separate in documentation and access control.
Pre-deployment validation
- Assemble a retrospective local sample covering the specialties in scope.
- Define acceptance criteria before looking at results — for example, ranked-option clinical appropriateness reviewed by specialty leads, network accuracy against the plan's authoritative file, and access predictions compared with realized time to appointment.
- Evaluate outcomes by segment: language, coverage type, geography, and age band.
- Review a stratified sample of recommendations with clinicians, including cases the model ranked poorly.
- Record the model version, data vintage, evaluation dataset, results, and the approval decision.
- Re-run validation whenever the model, the input set, or the network materially changes.
Ongoing monitoring dashboard
| Metric | Cadence | Escalation trigger (set locally) |
|---|---|---|
| Recommendation acceptance rate | Weekly | Sharp change without a known cause |
| Override rate and top reason codes | Weekly | Clinical-appropriateness overrides rising |
| Time to appointment by segment | Monthly | Gap widening between segments |
| In-network completion by segment | Monthly | Segment falls below defined floor |
| Patient decline rate | Monthly | Concentrated in one destination or population |
| Input freshness and completeness | Daily | Any file past its freshness threshold |
| Input and output distribution drift | Monthly | Statistically material shift |
| Unexplained recommendations | Continuous | Any occurrence |
| Open governance findings | Monthly | Any high severity past due |
Thresholds are organization-specific. Publish them, and require a written response when one is crossed.
Override and appeal workflow
- Clinician or coordinator selects a destination other than the top recommendation.
- System captures a structured reason: clinical judgment, patient preference, established relationship, access, transportation, language, or other with free text.
- Referral proceeds without additional approval steps.
- Override is logged with model version and displayed rationale.
- Patterns are reviewed monthly by clinical and network leadership.
- Patients who decline a recommended destination are offered alternatives, and the decline is recorded.
- A named appeal path exists for providers who believe network or access data about them is wrong, with a service-level target for correction.
Overrides are signal, not failure. A system with a zero override rate is usually a system nobody trusts enough to argue with.
Incident response and rollback
Referral matching incidents are rarely dramatic; they are usually quiet degradations that persist because nobody defined them as incidents. Establish severity levels in advance.
- High severity: recommendations produced from stale network or eligibility data, a prohibited input reaching the model, missing or incorrect explanations, or a segment-level access gap crossing an escalation threshold. Containment is immediate: suspend automated ranking and fall back to manual selection while preserving the workflow.
- Medium severity: input freshness breaches without evidence of patient impact, unexplained shifts in acceptance or override rates, or result-matching failures above threshold. Containment is a time-boxed fix with daily monitoring.
- Low severity: cosmetic or reporting defects with no decision impact, tracked in the standard backlog.
Every severity level needs a named on-call owner, a documented rollback path to the prior model and data version, a customer-facing notification commitment from the vendor, and a post-incident review that either closes a control gap or explains why none existed. Record incidents in the governance report even when patient impact was not identified — the absence of impact is a finding, not a reason to omit the entry.
Vendor change control
Model changes are the most common way a validated system becomes an unvalidated one. Contract for advance written notice of model, input, or scoring changes; a change log the customer can read; a defined re-validation trigger; and the ability to remain on a prior version for a stated period while local validation completes. Silent model updates should be treated as a contractual defect, not an operational surprise.
Procurement evidence list
Ask for these artifacts, in writing, before contract signature:
- Intended-use statement and documented limitations.
- Complete input register with provenance and refresh cadence.
- Sample per-recommendation explanation from a live screen.
- Override workflow demonstration and reason-code taxonomy.
- Validation methodology and evidence of local validation support.
- Fairness and drift monitoring approach, including what the customer can see.
- Audit log schema, retention, and export capability.
- Incident severity definitions, notification commitments, and rollback capability.
- Model change-control policy with customer notification terms.
- Security documentation, access model, and applicable agreements.
- Named accountable contacts for clinical safety and AI governance.
Where ReferralPoint fits
ReferralPoint's public documentation describes IdealMATCH™ as selecting in-network specialists using clinical need, network status, cost, quality, access, geography, language and social needs, and patient needs, with the selection connected to Auto PriorAUTH™ for authorization, Auto ReferralCOORDINATOR™ for outreach and scheduling, IntelligentDATA™ for the provider and network data layer, NetworkMANAGEMENT™ for network performance, and Auto 360° VISIBILITY™ for status and loop closure. Recommendations are ranked and clinician and patient choice are preserved. That input set maps directly to controls 2, 4, 6, and 9 above, which is why we encourage buyers to review it line by line. For security posture and integration controls, see integration and security and solutions for payers. Specific log schemas, monitoring views, and notification terms should be confirmed during procurement.
Sources and methodology
Control structure draws on primary and official sources: the NIST AI Risk Management Framework for govern, map, measure, and manage functions; HHS Office for Civil Rights materials on nondiscrimination and HIPAA privacy and security rules for access, minimum necessary, and safeguards; CMS materials on value-based care and the Closing the Referral Loop electronic clinical quality measure for outcome definitions; and ONC/ASTP certification program requirements addressing transparency for predictive decision support interventions. Vendor capability statements reflect official ReferralPoint documentation as of publication. Nothing here is legal advice; consult counsel on regulatory obligations specific to your organization.
Frequently asked questions
Q: How should healthcare organizations govern AI-based referral recommendations? A: Document the intended purpose and prohibited uses, approve a specific input register, require an explanation for every recommendation, keep human override immediate and logged, validate on local data before deployment, monitor fairness and drift on a published cadence, log every recommendation immutably, and review results in a named governance committee on a fixed schedule with written decisions.
Q: Can cost be an input to referral matching? A: Relative cost can be a legitimate input in value-based arrangements, alongside clinical fit, quality, and access. Two limits matter: it should never override clinical appropriateness or patient choice, and contracted rate detail generally does not belong on clinician-facing screens. Payer margin on the encounter and clinician compensation are not appropriate matching inputs at all.
Q: Should race or ethnicity ever be used in matching? A: Not as a ranking input. Those attributes are appropriate for fairness monitoring — checking whether access and completion differ across populations — but using them to rank destinations invites discriminatory routing. Keep the monitoring use and the matching use separated in documentation and access control, and assess granular geography and similar variables for proxy effects before enabling them.
Q: What does explainability mean in practice here? A: Not a research paper. At the moment of decision the user should see the ranked factors that produced the recommendation — network status, clinical fit, measured access, quality source, distance, language capability, patient preference — with enough detail to disagree. A model card supports procurement review; per-recommendation rationale is what makes override and audit meaningful.
Q: How often should we re-validate the model? A: Re-validate on a fixed schedule, typically annually, and additionally whenever the vendor changes the model, the input set changes, the network changes materially, or monitoring crosses an escalation threshold. Record the model version, data vintage, evaluation method, results, and approval decision every time so the audit trail shows which version governed which recommendations.
Q: What belongs in the audit log? A: For each recommendation: timestamp, model and data version, the inputs used, the ranked output presented, the explanation displayed, the user's action, any override reason code, and the eventual referral outcome. Logs should be immutable, retained per policy, exportable by the customer, and access-controlled. Ask vendors for the log schema before signing.
Q: Who should own AI governance for referrals? A: A standing committee with clinical leadership accountable for appropriateness, compliance accountable for permitted inputs and privacy, data or analytics accountable for validation and monitoring, security accountable for logging and access, and network operations accountable for vendor change control. One accountable owner per control. Monitoring nobody reads is not governance.
