CONCLAVE

The problem

Clinical genetics classifies variants (pathogenic → benign) against the ACMG/AMP guidelines: dozens of evidence criteria, each requiring literature review and judgement. LLMs can read the literature at scale — but an LLM that confidently guesses on a clinical criterion is worse than useless. The interesting problem is knowing when not to answer.

The design

CONCLAVE — Calibrated Orchestrated Network of Criterion-level LLM Agents for Variant Evaluation — decomposes interpretation into one agent per ACMG/AMP criterion, orchestrated into a final classification:

• Per-criterion conformal abstention — each agent's outputs are calibrated with conformal prediction, so a criterion abstains rather than answer outside its statistical confidence guarantee.
• Stratified calibration against ClinGen Variant Curation Expert Panel decisions, so calibration holds across variant types rather than just on average.
• Citation-faithfulness verification — every claim an agent makes is checked against the literature it cites, attacking the hallucination problem at the evidence level.
• Full provenance — classifications trace back to specific criteria, evidence and citations.

Engineering rigour

The repository is run like a research codebase: a project specification, architecture decision records, a model card, a system card, a dataset datasheet, reproducibility instructions and a calibration-debt log, with uv-managed environments and a pytest suite.

Status and limitations

The project is pre-alpha and is a research/portfolio project: I am not a geneticist, and its outputs must not be used for direct clinical decision-making. That boundary is documented in the system card — stating what a system must not be used for is part of building it responsibly.