Clinical Ops Copilot

Problem

Before a pharmacy can fill many medications, clinic staff have to prove to the insurer that the patient meets its coverage criteria: digging through the chart, filling out forms, faxing documentation. The American Medical Association reports practices complete dozens of these prior-authorization requests per physician every week, and every delay or denial pushes back patient care. Most of the work is mechanical (find the A1C, confirm the patient tried metformin, check the BMI against the payer's checklist), but the mechanical part is exactly where careless automation is dangerous: a system that confidently guesses when it's unsure is worse than the manual process it replaces.

Why it mattered

This is the project I built to answer the question a customer-facing AI role actually asks: can you build something that touches a real workflow, measure where it fails, and put a human in charge of every consequential action? It's a deployed, synthetic-data demo, not a production system, and I never call it one. But every layer, from live FHIR ingestion to the X12 wire format payers actually use, is built to the standard of "would I show a founder the failure modes, not just the demo."

Architecture

• The Approval UI (FastAPI + HTMX) is the only place a state-changing action can originate. It talks to the Agent, which owns the planner, guardrails, and audit logging.
• The Agent reaches two separate MCP servers: clinical-data (read-side: chart extraction, payer policy, FHIR client to HAPI FHIR and Synthea bundles, deployed on Fly.io over StreamableHTTP with bearer auth) and clinic-ops (action-side: send_email, create_task, run locally over stdio).
• A deterministic guardrail sits between the planner and the approval queue. If a required field is null or needs_review, the case is forced to request-more-info regardless of what the model would have guessed.
• The human approval gate is the last stop before anything leaves the system. No email is sent and no task is created until a staff member reviews it in the UI.
• The X12 278 layer sits at the payer-format edge: a hand-rolled parser reads an inbound 278 REQUEST into the agent's existing Case input, and a generator emits a 278 RESPONSE from the agent's decision (submit → A1, request-more-info or deny-risk → A4, never A3).
• OCR intake and a Playwright browser agent sit at the two intake/egress edges a deployment engineer actually touches: reading scanned decision letters, and reading authorization status back out of a payer portal (a self-built, clearly-labeled synthetic portal).
• Two more payer-format edges mirror the 278 layer: a self-authored X12 835 remittance parser feeding a deterministic denial-triage layer, and an HL7 v2 ingestion layer (ADT^A01 and ORU^R01) that maps messages into the agent's existing FHIR fact-resolution path.
• A real-telephony layer sits at the phone edge: a Twilio Programmable Voice webhook transcribes a spoken case number and question, routes it to the unchanged agent path, and speaks the decision back, with X-Twilio-Signature validation on every request.

Technical decisions

Two MCP servers instead of one. Alternative: a single MCP server handling both reads and actions. I split them because reads and actions have different trust boundaries: the read-side (clinical-data) can deploy publicly on Fly.io, while every action-side call has to pass the human gate first. Keeping them separate made that boundary structural instead of a rule I had to remember to enforce in code.

A deterministic guardrail on top of the model's own judgment. Alternative: trust the planner's prompting to ask for missing fields on its own. The pre-fix baseline had deny-risk recall of 0.600, meaning the model missed two out of five real denial risks. Adding a guardrail that routes any case with a null required field to request-more-info, independent of what the model would have said, is what moved that recall to 1.000.

Built an LLM-as-judge, then excluded it. Alternative: keep the judge in the scoring pipeline like most eval setups do. I validated it against 8 human ratings first: 0% exact agreement, MAE 1.38, and a Pearson correlation of about -0.29, meaning it disagreed with humans more often than chance would predict. I excluded it entirely rather than quietly blending a broken score into the eval.

The X12 278 response generator simulates the payer side rather than claiming to be one. Alternative: only build the request-parsing half and skip the response. Without generating a response there's no way to close the loop and test decision agreement, but claiming the agent issues real utilization-review determinations would misrepresent the demo. The response generator is explicitly framed as a simulation of what a payer-side determination would look like given the agent's own assessment.

Chaos-tested idempotency before adding more automation. Alternative: assume retries are safe by default. Under 30% injected failure, I verified 40/40 idempotent send_email calls produced exactly 40 backend sends (no doubles) and 20/20 action bundles completed once per key, closing the double-send failure mode before building anything else on top of the action layer.

Interesting challenges

The centerpiece failure is the LLM-as-judge. It's a common pattern to add a model-graded score to an eval pipeline, and it's tempting to trust it because it looks like more signal. When I checked it against 8 human ratings, it didn't just fail to agree, it correlated negatively with human judgment. I excluded it from scoring and kept the locked, human-labeled eval as the only source of truth. That's the honest-evals story I'd tell in an interview before anyone asks.

The deny-risk to A4 mapping was a smaller but deliberate decision: the automated layer never issues an A3 (denied). A likely-denial flag is pended for human review, not adjudicated, because only a human downstream has the authority to deny. On the locked eval split, the offline decider produced 12 submit and 4 request-more-info cases and zero deny-risk cases, so that specific mapping is covered by unit tests, not by the held-out eval; I disclose that gap rather than let the 16/16 number imply more than it measures.

The OCR layer's honest surprise: raw tesseract genuinely garbles degraded scans (on one letter, "Case ID" reads as "Case 1D"), and the tolerant parser recovers the field anyway through case-insensitive matching and light noise remapping. The 96/96 field accuracy is real, but it's 100% on a 12-letter set I generated and scored myself, not a claim about real payer letters.

Live telephony

The copilot also answers a real phone call. A caller dials a Twilio number, Twilio Programmable Voice transcribes the speech, a FastAPI TwiML webhook routes the case number to the unchanged agent decision path, and the decision is spoken back on the call. The telephony layer only routes and phrases; the agent makes the whole decision, exactly as it does from the command line. I verified this end to end on 2026-07-16 with a real inbound call: I spoke a case number and a question, and I heard the agent's spoken decision back.

Two design details carry the section.

Every webhook request is signature-validated. I hand-rolled the X-Twilio-Signature check, implementing Twilio's documented HMAC-SHA1 algorithm directly so the security-critical path has no runtime dependency on the Twilio SDK and stays fully testable offline. A test cross-checks it byte for byte against the official SDK validator, the comparison is constant-time, and there is no bypass path. During the live session, every unsigned probe was rejected with a 403.

Hold-and-poll under a hard 15-second deadline. A full agent decision takes roughly as long as Twilio's 15-second webhook timeout, so answering synchronously makes Twilio give up and play an error message. The first request instead starts the agent as a background task keyed by the call id, returns an immediate spoken hold message, and redirects Twilio to poll back; the decision is read on whichever poll finds the task finished. Only the timing of when the answer is spoken changes, and the agent path stays untouched.

This is a demo on the parent repo's synthetic cases, routed by case number only. It is not a call-center system and it is not telephony at scale.

Claims-lifecycle interop

The X12 278 prior-auth layer is one of three demo-scope interop layers that let the copilot speak the formats payers and hospitals actually move data in. Two more sit alongside it, each built on the same hand-rolled tokenizer and error core, and each on synthetic self-authored data.

X12 835 remittance parser and denial-triage layer. A self-authored X12 835 (claim payment and remittance advice) subset parses into a typed RemittanceAdvice: claims, billed versus paid, and denial codes. A deterministic denial-triage layer, with no model in the loop and a recommendation that is a pure function of the codes, then routes each claim to one of four actions: no-action, resubmit-with-documentation, correct-and-rebill, or needs-human-review. It uses a conservative precedence and two fail-safe-to-human paths, one for an unrecognized code and one for an underpayment with no coded reason. It reaches 9/9 exact-match with precision and recall of 1.000 per class on its 9-claim self-authored fixture set. The denial codes are an invented DR-* system carried in an invented DRC segment, not real CARC or RARC content, and the data is synthetic, so this is a remittance-review simulation rather than real denial management.

HL7 v2 ingestion layer. Most hospital feeds still move over HL7 v2 pipe-and-caret messages rather than FHIR. A dependency-free HL7 v2 ingestion layer parses the admit and observation-result message types (ADT^A01 and ORU^R01), resolves the delimiters from the MSH header, handles HL7 escape sequences, and maps the result into the copilot's existing FHIR fact-resolution path. It scores 6/6 exact-match on its self-authored synthetic HL7 v2 set, so the number measures ingestion fidelity and regression, never real-world interoperability. In the same honest-evals spirit as the rest of the project, the mapping is additive: a diff of the agent, schema, and FHIR-client code against main comes back empty, which proves the existing decision path was byte-untouched. This is a demo-scope subset, not a certified interface engine.

Results

Lessons

I would have shipped a broken scorer if I hadn't validated the judge against real human ratings before trusting it. Now I check any model-graded metric against a small human-labeled sample before it goes anywhere near a scoreboard.

Guardrails need to be deterministic when the failure mode is a missed field, not just a well-prompted model. The recall jump from 0.600 to 1.000 came from code, not a better prompt.

Simulating both sides of a protocol (parsing the request, generating the response) taught me how easy it is to accidentally overclaim what a demo does. Framing the response generator as a simulation, explicitly, was the difference between an honest artifact and a misleading one.

Future work

• Grow the locked eval split past n=16 for tighter confidence intervals on the decision metrics.
• Exercise the deny-risk to A4 mapping in the held-out eval itself, not only in unit tests.
• Test the X12 278 layer against a real clearinghouse sandbox instead of only a self-consistency round-trip.