# Can Language Models Be Trusted With Clinical Measurement?

A talk by Ming-Chih Kao, PhD, MD, FIPP, CIPS. An AI that quotes a cutoff from the wrong population gives a confident, wrong answer. Grounding, attribution and refusal matter more than fluency.

Clinicians and patients increasingly ask AI systems what a score means. Dr. Kao, a biostatistician and pain physician, presents two projects that address the problem. MeasureGPT is a psychometric interpretation service that gives AI systems verified severity bands, T-scores and percentiles, and says plainly when no defensible reference population exists. ESTIMAND is a benchmark of grounded psychometric reasoning: whether a model can retrieve, attribute and compute with a measurement fact, recognize when the fact does not exist, and judge whether it applies to the patient at hand.

## Attendees will be able to

1. Show where language models fail when they interpret questionnaires and scores
2. Explain why a psychometric parameter is only valid in the population that produced it
3. Demonstrate a grounded approach that serves verified measurement facts to AI systems over the Model Context Protocol
4. Outline how a benchmark can test attribution, computation and knowing when not to answer

## Formats

Keynote, Panel, Grand rounds; 20, 30, 45 minutes.

## For

- AI in medicine and digital health meetings
- Informatics and data science audiences
- Health system AI governance committees

To invite: use the contact form at https://drmingkao.com/contact/?reason=speaking&talk=trustworthy-ai-measurement.
