Identifies and operationalises cultural confabulation — a previously uncharacterised failure mode where models produce contextually invalid reasoning by substituting dominant cultural priors for missing lived context. Introduces the gap proof: the delta between scaffolded and unscaffolded performance as a measure of structural failure.
Published · TAIS 2026 ProceedingsMulti-epoch empirical validation of the scaffolding gap across three frontier models and three evaluation domains (N=72 observations). Establishes cross-model grading methodology and quantifies systematic grader bias across model families.
Working Paper · 2026Identifies structural issues in model-as-judge evaluation pipelines: when the grader shares the same priors as the model under evaluation, failure modes are systematically masked.
Working Paper · 2026Current biosecurity evaluations test whether AI systems are dangerous. This framework tests whether AI systems understand that they could be dangerous — and reason accordingly.
Working Paper · 2026Collaborative Delphi-style consensus publication for Lancet Primary Care on evaluation and deployment standards for AI in community health settings. Research collaboration with the Community Health Impact Coalition (CHIC), spanning 20+ countries with clinicians, ministry-of-health stakeholders, and WHO advisors.
In Progress · Lancet Primary CareProgramme committee reviewer. International Conference on Machine Learning, Vancouver.
July 2026Invited speaker. Presentation on cultural confabulation and contextual validity in frontier models.
May 2026Invited moderator. Roundtable on AI evaluation and evidence standards for community health worker programmes. Convened by the Community Health Impact Coalition (CHIC).
April 2026Invited panellist, with Tom Shakespeare and MP Liam Conlon.
December 2025Four-domain evaluation framework for frontier LLM deployment in global health and biosecurity contexts. Built on UK AISI Inspect. Tested across Claude Sonnet 4, GPT-4o, and Gemini 2.5 Pro (N=72 observations, Cohen's d 0.95–1.82).
GitHub Repository