AlignmentTailBench
Lightweight eval harness for ranking prompts by reward variance, CVaR, lower-tail reward, and failure examples.
Repository coming soonHsiang Hsu
Research Scientist · Frontier Model Evals, Alignment Robustness & Trustworthy AI
I work on stress-testing model behavior beyond average-case evaluation: residual knowledge after unlearning, prompt-specific reward tail risk, predictive multiplicity, and hidden failure modes in generative systems. My goal is to turn uncertainty, privacy, and robustness failures into operational evals for model behavior, alignment, and intervention verification.
Research
I care about building reliable evaluation methods for frontier models and AI systems: tests that expose where models are brittle, unstable, or unsafe even when aggregate metrics look strong. Current and previous threads include:
- Frontier model evals and alignment robustness — reward tail risk, prompt-level instability, and inference-time alignment failures.
- Machine unlearning and intervention verification — detecting residual knowledge that reappears under perturbations or distribution shift.
- Predictive multiplicity and decision uncertainty — measuring arbitrary decisions among near-equivalent models.
- Privacy, fairness, and information-theoretic ML — robustness, attribute protection, and governance without relying on sensitive labels.
Research Artifacts
ResidualKnowledgeProbe
Stress test for unlearning and model editing: finds cases where models pass forgetting checks but recover knowledge under perturbations.
Repository coming soonBehavioralMultiplicity
Demo for surfacing unstable individual decisions across near-equivalent models or checkpoints with similar aggregate performance.
Repository coming soonSelected Papers
Frontier Model Evals, Unlearning, and Generative Model Safety
- The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples NeurIPS 2025
Shows that models can appear to forget under standard tests while retaining residual knowledge under perturbed deletion queries.
- HeavyWater and SimplexWater: Distortion-Free LLM Watermarks for Low-Entropy Next-Token Predictions NeurIPS 2025
Studies watermarking behavior in low-entropy LLM regimes where naive watermarking may be brittle or distortive.
- PaLD: Detection of Text Partially Written by Large Language Models ICLR 2025
Detects hybrid human/LLM text rather than assuming documents are fully human- or machine-written.
- Machine Unlearning for Image-to-Image Generative Models ICLR 2024
Develops unlearning methods for generative image-to-image models.
Predictive Multiplicity, Decision Uncertainty, and Alignment Robustness
- RashomonGB: Analyzing the Rashomon Effect and Mitigating Predictive Multiplicity in Gradient Boosting NeurIPS 2024
Analyzes conflicting individual predictions among similarly accurate gradient boosting models.
- Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation ICLR 2024
Uses dropout-based exploration to estimate model multiplicity more efficiently.
- Individual Arbitrariness and Group Fairness NeurIPS Spotlight 2023
Studies how group-level fairness constraints can induce individual-level arbitrariness.
- Rashomon Capacity: A Metric for Predictive Multiplicity in Classification NeurIPS 2022
Introduces a metric for quantifying how much near-optimal models disagree on individual decisions.
- Best-of-Tails: Bridging Optimism and Pessimism in Inference-Time Alignment arXiv preprint
Frames prompt-specific reward distributions and tail outcomes as signals for inference-time alignment risk.
Fairness, Privacy, and Information-Theoretic ML
- PASS: Private Attributes Protection with Stochastic Data Substitution ICML Spotlight 2025
Protects private attributes while preserving downstream utility through stochastic data substitution.
- MaSS: Multi-attribute Selective Suppression for Utility-preserving Data Transformation ICML 2024
Uses information-theoretic transformations to suppress multiple protected attributes while retaining utility.
- Beyond Adult and COMPAS: Fair Multi-Class Prediction via Information Projection NeurIPS Oral 2022
Develops fairness interventions for multi-class prediction using information projection.
- Generalizing Correspondence Analysis for Applications in Machine Learning TPAMI 2021
Extends correspondence analysis for representation and machine-learning applications.
Experience
Research Lead · Applied Intelligence Lab
Leading work on adversarial evaluation, unlearning verification, reward and decision instability, and model safety governance.
Research Scientist · Applied Intelligence Lab
Developed evaluation methods for residual knowledge, LLM watermarking, partial-LLM text detection, hallucination probing, and predictive multiplicity.
Research Assistant
Worked on information-theoretic tools for fairness, privacy, representation learning, and decision uncertainty.
Recognition & Service
- Meta PhD Fellowship in Applied Statistics, selected from 2,000+ applicants.
- NeurIPS Oral, NeurIPS Spotlight, and ICML Spotlight distinctions.
- JPMC Prolific Inventor for 10+ U.S. patent filings in trustworthy AI, model safety, and governance.
- Program/area chair and reviewer for FAccT, ICML, NeurIPS, ICLR, AISTATS, ISIT, TIT, TMLR, and TheWebConf.
Education
Harvard University
Thesis: Information-Theoretic Tools for Machine Learning Beyond Accuracy — fairness, privacy, and decision uncertainty.
National Taiwan University
Best Master’s Thesis Award and National Young Scholar Best Paper Award.
Selected Media
- The Daily Upside — machine unlearning and privacy guardrails.
- UT News, The Daily Texan, PetaPixel — generative machine unlearning.
- Meta Research Spotlight — Rashomon effect and predictive multiplicity.