Jihye Choi

Research Scientist, Security and Privacy @ NVIDIA Research

Email: <firstname><last initial> AT nvidia DOT com
Google Scholar · GitHub · LinkedIn · CV

I’m a Research Scientist in the Security and Privacy Research group at NVIDIA Research, led by Edward Suh. I received my Ph.D. in Computer Sciences from the University of Wisconsin-Madison, advised by Somesh Jha.

My current research focuses include, but are not limited to:

Safety and security of agents: making agents themselves safe and secure, at the model level (post-training, alignment) and at the system level (guarantees, policy constraints, verification, sandboxing).
Agents for security: making agents better at security tasks by recursively self-improving the model or the agent harness, in AI security (e.g., defenses against prompt injection) and cybersecurity (e.g., discovering, verifying, and patching software vulnerabilities).

I’m always happy to chat about research. If you’d like to collaborate, feel free to reach out!

Publications

Formal Policy Enforcement for Real-World Agentic Systems
Nils Palumbo, Sarthak Choudhary, Jihye Choi, Guy Amir, Prasad Chalasani, Somesh Jha
Preprint | BibTeX

@misc{palumbo2026formal,
  title={Formal Policy Enforcement for Real-World Agentic Systems},
  author={Palumbo, Nils and Choudhary, Sarthak and Choi, Jihye and Amir, Guy and Chalasani, Prasad and Jha, Somesh},
  year={2026},
  eprint={2602.16708},
  archivePrefix={arXiv},
  primaryClass={cs.CR},
  url={https://arxiv.org/abs/2602.16708}
}

Agent Security is a Systems Problem
Mihai Christodorescu, Earlence Fernandes, Ashish Hooda, Somesh Jha, Johann Rehberger, Kamalika Chaudhuri, Xiaohan Fu, Khawaja Shams, Guy Amir, Jihye Choi, Sarthak Choudhary, Nils Palumbo, Andrey Labunets, Nishit V. Pandya
NeurIPS 2026 (oral) | BibTeX

@inproceedings{christodorescu2026agent,
  title={Agent Security is a Systems Problem},
  author={Christodorescu, Mihai and Fernandes, Earlence and Hooda, Ashish and Jha, Somesh and Rehberger, Johann and Chaudhuri, Kamalika and Fu, Xiaohan and Shams, Khawaja and Amir, Guy and Choi, Jihye and Choudhary, Sarthak and Palumbo, Nils and Labunets, Andrey and Pandya, Nishit V.},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2026}
}

PolicyBank: Evolving Policy Understanding for LLM Agents
Jihye Choi, Jinsung Yoon, Long T. Le, Somesh Jha, Tomas Pfister
COLM 2026 | BibTeX

@inproceedings{choi2026policybank,
  title={PolicyBank: Evolving Policy Understanding for LLM Agents},
  author={Choi, Jihye and Yoon, Jinsung and Le, Long T. and Jha, Somesh and Pfister, Tomas},
  booktitle={Conference on Language Modeling (COLM)},
  year={2026}
}

ATLAS: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning
Jihye Choi, Jinsung Yoon, Jiefeng Chen, Somesh Jha, Tomas Pfister
ICLR 2026 | Demo | BibTeX

@inproceedings{choi2026atlas,
  title={{ATLAS}: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning},
  author={Jihye Choi and Jinsung Yoon and Jiefeng Chen and Somesh Jha and Tomas Pfister},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026}
}

MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance
Jihye Choi*, Nils Palumbo*, Prasad Chalasani, Matthew M. Engelhard, Somesh Jha, Anivarya Kumar, David Page
MLHC 2024 | Github | Blog | BibTeX

@inproceedings{choi2024malade,
  title={{MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance}},
  author={Jihye Choi and Nils Palumbo and Prasad Chalasani and Matthew M. Engelhard and Somesh Jha and Anivarya Kumar and David Page},
  booktitle={Proceedings of the Machine Learning for Healthcare Conference (MLHC)},
  year={2024},
  url={https://arxiv.org/abs/2408.01869}
}

Agent Security is a Systems Problem
Mihai Christodorescu, Earlence Fernandes, Ashish Hooda, Somesh Jha, Johann Rehberger, Kamalika Chaudhuri, Xiaohan Fu, Khawaja Shams, Guy Amir, Jihye Choi, Sarthak Choudhary, Nils Palumbo, Andrey Labunets, Nishit V. Pandya
NeurIPS 2026 (oral) | BibTeX

@inproceedings{christodorescu2026agent,
  title={Agent Security is a Systems Problem},
  author={Christodorescu, Mihai and Fernandes, Earlence and Hooda, Ashish and Jha, Somesh and Rehberger, Johann and Chaudhuri, Kamalika and Fu, Xiaohan and Shams, Khawaja and Amir, Guy and Choi, Jihye and Choudhary, Sarthak and Palumbo, Nils and Labunets, Andrey and Pandya, Nishit V.},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2026}
}

PolicyBank: Evolving Policy Understanding for LLM Agents
Jihye Choi, Jinsung Yoon, Long T. Le, Somesh Jha, Tomas Pfister
COLM 2026 | BibTeX

@inproceedings{choi2026policybank,
  title={PolicyBank: Evolving Policy Understanding for LLM Agents},
  author={Choi, Jihye and Yoon, Jinsung and Le, Long T. and Jha, Somesh and Pfister, Tomas},
  booktitle={Conference on Language Modeling (COLM)},
  year={2026}
}

SLVR: Securely Leveraging Client Validation for Robust Federated Learning
Jihye Choi, Rahul Rachuri, Ke Wang, Somesh Jha, Yizhen Wang
AsiaCCS 2026 Workshop on Secure and Efficient Federated Learning | BibTeX

@inproceedings{choi2026slvr,
  title={SLVR: Securely Leveraging Client Validation for Robust Federated Learning},
  author={Choi, Jihye and Rachuri, Sai Rahul and Wang, Ke and Jha, Somesh and Wang, Yizhen},
  booktitle={Proceedings of the International Workshop on Secure and Efficient Federated Learning (FL '26)},
  pages={1--6},
  year={2026},
  publisher={ACM},
  doi={10.1145/3803630.3809167}
}

ATLAS: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning
Jihye Choi, Jinsung Yoon, Jiefeng Chen, Somesh Jha, Tomas Pfister
ICLR 2026 | Demo | BibTeX

@inproceedings{choi2026atlas,
  title={{ATLAS}: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning},
  author={Jihye Choi and Jinsung Yoon and Jiefeng Chen and Somesh Jha and Tomas Pfister},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026}
}

Formal Policy Enforcement for Real-World Agentic Systems
Nils Palumbo, Sarthak Choudhary, Jihye Choi, Guy Amir, Prasad Chalasani, Somesh Jha
Preprint | BibTeX

@misc{palumbo2026formal,
  title={Formal Policy Enforcement for Real-World Agentic Systems},
  author={Palumbo, Nils and Choudhary, Sarthak and Choi, Jihye and Amir, Guy and Chalasani, Prasad and Jha, Somesh},
  year={2026},
  eprint={2602.16708},
  archivePrefix={arXiv},
  primaryClass={cs.CR},
  url={https://arxiv.org/abs/2602.16708}
}

MAP: Multi-user Personalization with Collaborative LLM-powered Agents
Christine P. Lee*, Jihye Choi*, Bilge Mutlu
CHI 2025 (LBW) | Github | BibTeX

@article{lee2025map,
  title={MAP: Multi-user Personalization with Collaborative LLM-powered Agents},
  author={Lee, Christine and Choi, Jihye and Mutlu, Bilge},
  journal={arXiv preprint arXiv:2503.12757},
  year={2025}
}

CONDA: Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts
Jihye Choi, Jayaram Raghuram, Yixuan Li, Somesh Jha
ICLR 2025 | Github | Poster | BibTeX

@inproceedings{choi2025conda,
  title={Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts},
  author={Jihye Choi and Jayaram Raghuram and Yixuan Li and Somesh Jha},
  booktitle={The Thirteenth International Conference on Learning Representations},
  year={2025},
  url={https://openreview.net/forum?id=8sfc8MwG5v}
}

MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance
Jihye Choi*, Nils Palumbo*, Prasad Chalasani, Matthew M. Engelhard, Somesh Jha, Anivarya Kumar, David Page
MLHC 2024 | Github | Blog | BibTeX

@inproceedings{choi2024malade,
  title={{MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance}},
  author={Jihye Choi and Nils Palumbo and Prasad Chalasani and Matthew M. Engelhard and Somesh Jha and Anivarya Kumar and David Page},
  booktitle={Proceedings of the Machine Learning for Healthcare Conference (MLHC)},
  year={2024},
  url={https://arxiv.org/abs/2408.01869}
}

PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails
Neal Mangaokar*, Ashish Hooda*, Jihye Choi, Shreyas Chandrashekaran, Kassem Fawaz, Somesh Jha, Atul Prakash
ACL 2024 | Github | BibTeX

@inproceedings{mangaokar2024prp,
  title={{PRP}: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails},
  author={Mangaokar, Neal and Hooda, Ashish and Choi, Jihye and Chandrashekaran, Shreyas and Fawaz, Kassem and Jha, Somesh and Prakash, Atul},
  booktitle={Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year={2024},
  pages={10960--10976},
  url={https://aclanthology.org/2024.acl-long.591}
}

Why Train More? Effective and Efficient Membership Inference via Memorization
Jihye Choi, Shruti Tople, Varun Chandrasekaran, Somesh Jha
Preprint | BibTeX

@article{choi2023train,
  title={Why train more? effective and efficient membership inference via memorization},
  author={Choi, Jihye and Tople, Shruti and Chandrasekaran, Varun and Jha, Somesh},
  journal={arXiv preprint arXiv:2310.08015},
  year={2023}
}

Identifying and Mitigating the Security Risks of Generative AI
Clark Barrett, Brad Boyd, Elie Bursztein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, Kathleen Fisher, Tatsunori Hashimoto, Dan Hendrycks, Somesh Jha, Daniel Kang, Florian Kerschbaum, Eric Mitchell, John Mitchell, Zulfikar Ramzan, Khawaja Shams, Dawn Song, Ankur Taly, Diyi Yang
FnT in Privacy and Security | Workshop | BibTeX

@article{barrett2023identifying,
  title={Identifying and mitigating the security risks of generative ai},
  author={Barrett, Clark and Boyd, Brad and Bursztein, Elie and Carlini, Nicholas and Chen, Brad and Choi, Jihye and Chowdhury, Amrita Roy and Christodorescu, Mihai and Datta, Anupam and Feizi, Soheil and others},
  journal={Foundations and Trends in Privacy and Security},
  volume={6}, number={1}, pages={1--52}, year={2023},
  publisher={Now Publishers, Inc.}
}

Concept-based Explanations for Out-of-Distribution Detectors
Jihye Choi, Jayaram Raghuram, Ryan Feng, Jiefeng Chen, Somesh Jha, Atul Prakash
ICML 2023 | Github | BibTeX

@inproceedings{choi2023concept,
  title={Concept-based explanations for out-of-distribution detectors},
  author={Choi, Jihye and Raghuram, Jayaram and Feng, Ryan and Chen, Jiefeng and Jha, Somesh and Prakash, Atul},
  booktitle={International Conference on Machine Learning},
  pages={5817--5837}, year={2023}, organization={PMLR}
}

Stratified Adversarial Robustness with Rejection
Jiefeng Chen*, Jayaram Raghuram*, Jihye Choi, Xi Wu, Yingyu Liang, Somesh Jha
ICML 2023 | Github | BibTeX

@inproceedings{chen2023stratified,
  title={Stratified Adversarial Robustness with Rejection},
  author={Jiefeng Chen and Jayaram Raghuram and Jihye Choi and Xi Wu and Yingyu Liang and Somesh Jha},
  booktitle={International Conference on Machine Learning},
  year={2023}
}

Rethinking Diversity in Deep Neural Network Testing
Zi Wang, Jihye Choi, Ke Wang, Somesh Jha
Preprint | BibTeX

@article{wang2023rethinking,
  title={Rethinking Diversity in Deep Neural Network Testing},
  author={Wang, Zi and Choi, Jihye and Wang, Ke and Jha, Somesh},
  journal={arXiv preprint arXiv:2305.15698},
  year={2023}
}

Defending Object Detectors against Patch Attacks with Out-of-Distribution Smoothing
Ryan Feng, Neal Mangaokar, Jihye Choi, Somesh Jha, Atul Prakash
Preprint | BibTeX

@misc{feng2024patch,
  title={Defending Object Detectors against Patch Attacks with Out-of-Distribution Smoothing},
  author={Ryan Feng and Neal Mangaokar and Jihye Choi and Somesh Jha and Atul Prakash},
  year={2024}, eprint={2205.08989}, archivePrefix={arXiv}, primaryClass={cs.LG}
}

Revisiting Adversarial Robustness of Classifiers With a Reject Option
Jiefeng Chen*, Jayaram Raghuram*, Jihye Choi, Xi Wu, Yingyu Liang, Somesh Jha
AAAI 2022 Workshop on Adversarial Machine Learning and Beyond Best Paper Award | BibTeX

@inproceedings{chen2022revisiting,
  title={Revisiting adversarial robustness of classifiers with a reject option},
  author={Chen, Jiefeng and Raghuram, Jayaram and Choi, Jihye and Wu, Xi and Liang, Yingyu and Jha, Somesh},
  booktitle={The AAAI-22 Workshop on Adversarial Machine Learning and Beyond},
  year={2022}
}

Stochastic Doubly Robust Gradient
Kanghoon Lee*, Jihye Choi*, Moonsu Cha, Jung-Kwon Lee, Tae Yoon Kim
Preprint
Data-driven Approach to Aesthetic Enhancement
Jihye Choi, Sungjoon Koh, Jongwoo Kwack, Yonghun Kwon, Hyunjung Shim
SPIE Electronic Imaging 2016

Background

Employment

Research Scientist, NVIDIA Research (Security and Privacy Research), May 2026 – present
Manager: Edward Suh

Education

Ph.D. in Computer Sciences, University of Wisconsin–Madison
M.S. in Electrical and Computer Engineering, Carnegie Mellon University
B.S., School of Integrated Technology, Yonsei University

Experience

Research Intern, Google Cloud AI Research, May – Dec. 2025
With Dr. Jinsung Yoon
Graduate Research Assistant, University of Wisconsin–Madison, 2020 – 2025
With Prof. Somesh Jha
PhD Research Intern, Visa Research, Identity and AI Security, Summer 2022, Summer 2023
With Dr. Yizhen Wang, Dr. Ke Wang, Dr. Rahul Rachuri
Research Assistant, CyLab, Carnegie Mellon University, May 2017 – Mar. 2019
With Prof. Lujo Bauer
Research Intern, T-Brain, SK Telecom, Apr. 2018 – Sep. 2018
With Dr. Kanghoon Lee
Undergraduate Research Assistant, Vision & Learning Lab, Yonsei University, Mar. 2015 – Aug. 2016
With Prof. Hyunjung Shim

Service

Reviewing

Program Committee, AAAI 2027
Program Committee, Deep Learning Security and Privacy Workshop, co-located with IEEE S&P 2025
Student Organizer, GenAI Risk Workshop 2023
Reviewer for NeurIPS 2022–2024, 2026
Reviewer for ICLR 2026, 2027
Reviewer for COLM 2025, 2026
Reviewer for ICML 2024, 2025
Reviewer for CHI 2026
Reviewer for MLHC 2025
Reviewer for AISTATS 2025
Reviewer for CVPR 2025
Reviewer for ICCV 2025
External Reviewer for SaTML 2024
Reviewer for IEEE TPAMI 2025

Teaching

CS 435 Introduction to Cryptography, University of Wisconsin–Madison (Fall 2021, Fall 2020, Spring 2020)
Teaching Assistant with Prof. Somesh Jha
CS 412 Numerical Analysis, University of Wisconsin–Madison (Fall 2019)
Teaching Assistant with Prof. Amos Ron
18-290 Signals and Systems, Carnegie Mellon University (Spring 2017)
Head Teaching Assistant with Prof. Aswin C. Sankaranarayanan and Prof. Richard Stern

Mentoring

Graduate Mentor, Women in Science and Engineering (WISE), University of Wisconsin–Madison, 2025–2026
Graduate Mentor, Korean-American Scientists and Engineers Association (KSEA), Wisconsin Chapter, 2021–2022