Research Portfolio

Building Transparent Systems

I focus on the assurance of generative AI systems, from dynamic evaluation of retrieval architectures to mechanistic assurance and monitoring of reasoning systems. Below is a closer look at selected works and publications.

96

Citations

7

h-index

Google Scholar ↗
01

Recent Work

2025Completed

MERIT: Mechanistic Explainability of Reasoning Integrity and Transparency

Mechanistic interpretability framework for identification of domain-specific reasoning features over varying task complexities.

Developed six metrics to quantitatively track LLM reasoning output quality
Identified feature fracturing behavior into specialized sub-domains over increasing task difficulty
Discovered discrete feature-controlled reasoning modalities (e.g. calculative vs verbose) and demonstrated steering efficacy
Mechanistic InterpretabilityAI AssuranceLLM Reasoning
2025Completed

Resurgens

An AI system for reconstructing the Etruscan language using contrastive phonological modeling and unsupervised clustering.

Aligned Latin, Greek, and Etruscan data to hypothesize missing or uncertain forms
Implemented phonological constraints that guide generation toward linguistically plausible reconstructions
Built curated corpora tailored for low-resource modeling and evaluation
Computational LinguisticsLow-Resource NLPContrastive Modeling
2026 — PresentIn progress

Error Detection in AI-Generated Discharge Summaries

Detecting conditional errors in LLM-generated hospital discharge instructions, where summarizers keep an action but drop or alter its trigger (e.g. "call if fever exceeds 101°F").

Defined a six-type taxonomy of conditional errors and built a perturbation pipeline that generates training data with labels guaranteed by construction
Fine-tuned NLI detectors reaching 88.1% omission and 90.4% substitution accuracy on held-out real MIMIC-III documents at ~72 ms per document
Produced reusable clinical NLP infrastructure: 32,803 discharge-instruction sections, 8,000+ labeled sentences, and a clinical-aware sentence splitter
Medical NLPClinical SummarizationAI Assurance
02

Selected Writeups

2024MERIT: Mechanistic Explainability of Reasoning Integrity and TransparencyR Shan · Preprint2025Qwen 2.5 7B SAE TrainingR Shan · Technical Writeup2024Language Artificial Intelligence at a Crossroads: Deciphering the Future of Small and Large Language ModelsR Shan · IEEE Computer 57 (8), 26-35