Ask a leading foundation model to write a health report from someone’s continuous glucose data and meal log, and you get something fluent, confident and partly invented. On identical two weeks of CGM and meal data, the frontier models we compared, among them GPT-5.4, Claude Opus 4.6 and Gemini 3.1 Pro, invented glycaemic variability values, counted 172 meals where the log held 70, and projected complication risks that referenced nothing. None of it looks wrong. That is the problem: spotting the errors means going back to the source data and recomputing, and neither the person nor their clinician is in a position to do that.

I led the project to fix this, a five-person effort across teams at Pheno AI, from architecture through to evaluation.

What we built

The architecture is deliberately plain: one ReAct orchestrator over 21 tools and trained-model wrappers, in six groups.

  • CGM analysis, dietary quality and meal-glucose response compute the clinical numbers.
  • Predictive models return the output of models we trained, such as dysglycaemia risk and metabolic age, each carrying its own accuracy.
  • Population comparison places a person against the Human Phenotype Project reference cohort, which is where the percentiles come from.
  • Literature tools fetch sources.

So the agent has more than functions to call: it has trained models, and the cohort data to compare a person against. Every number in the report comes from those. The model decides what to ask for, interprets what comes back, and connects the results into a single report that holds together as an account of one person’s metabolic health.

The other half is context, in two kinds.

  • Curated context is 12 skills, similar to the skills that modern agents use: a markdown file with no code in it, injected into the system prompt. Each one declares which tools’ outputs it may refer to, then lists the specific fabrications it forbids. That list is not theoretical: one of our own skill examples showed time-of-day glucose patterns, which no tool computes, and taught the model to fabricate them.
  • Retrieved context is what the literature search brings back while the report is being written.

What it changed

On a deterministic score for form and provenance, which checks whether a report contains the parts it should and whether each number can be traced to the tool that produced it, the full system scores 0.91 against 0.37 for the baseline.

The baseline is not a weak model. It is the same frontier model with the tools, the trained models, the data and the skills taken away. On its own it scores low on most of the evals.

The gain then splits in two, because the two halves fail separately. Tools drive numerical accuracy: the share of reported clinical metrics that are correct rises from about 14% to 90%. Skills drive nearly everything else. Tools with no skills reach only 0.49, so roughly four fifths of the form and provenance gap is closed by the skills rather than by the tools.

A frontier model on its own scores low on almost every eval. The full system scores high on almost all of them. Tools alone lift numerical accuracy; citations, completeness, meal reporting and population comparison barely move until the skills go in.

A frontier model on its own scores low on almost every eval. The full system scores high on almost all of them. Tools alone lift numerical accuracy; citations, completeness, meal reporting and population comparison barely move until the skills go in.

What it still gets wrong

  • The evaluation measures form, not substance. It checks citations, tool calls, numerical accuracy, forbidden words and structure. Whether a report is clinically appropriate, or of any use to the person reading it, needs expert review and a prospective study.
  • Each of the 21 evals encodes a failure we had already found and fixed. A high pass rate says we closed the failures we knew about. It does not say the system is right on cases we have not seen.
  • We are not uniformly better than the baselines on every dimension. Their reports occasionally have more fluent prose, or more detailed behavioural advice, or pick out a qualitative pattern our tools do not compute, such as a dawn-phenomenon signature. That last one is a gap in the tool suite.

Where it goes next

The report is one-way. The next system is interactive: a Q&A agent over the same tools, and coaching on top of that. It also needs a prospective study with clinicians and participants, which is the only way to settle what the current evaluation cannot.

I am interested in the opposite direction too. Once the computational core is stable, much of it may not need an agent at all: deterministic pipelines can do the arithmetic, and the agent can be left to route and to write.

Read it

The preprint is on medRxiv, and the code is public: the architecture, how it works over multimodal personal health data, and the evaluation harness.