Kaiser Sun

Kaiser Sun


πŸ³οΈβ€πŸŒˆ they/them
hsun74 at jhu.edu

Hello! I am a PhD student at JHU CLSP and the JHU Data Science and AI Institute (DSAI), advised by Mark Dredze and Daniel Khashabi.

My interests are protean due to my restless curiosity. I am generally interested in understanding how language models function and exploring how we can change them. Terminologically, I am interested in studying interpretability (broadly), training dynamics, evaluation, and reasoning of language models.

Before my PhD studies, I was a resident at FAIR Labs, working with Adina Williams and Dieuwke Hupkes. I obtained my Master’s and Bachelor’s degrees of computer science (B.S., M.S.) and mathematics (B.A.) with a minor in classical studies at University of Washington. My advisor was Noah A. Smith, and I was supervised by Ana MarasoviΔ‡. I have also interned at the AWS AI Labs for two times, mentored by Peng Qi, Yuhao Zhang, Jifan Chen, and Danilo Ribeiro. In summer 2026, I interned at Together AI, working with Fede Bianchi and Kaitlyn Zhou. My work has been recognized with an Outstanding Paper Award at COLM2026, a Best Paper Honorable Mention at CoNLL2023, an Outstanding Paper Award at MLRC2022, and some media coverage. Thanks a lot to the support of my advisors and my donors, I was able to conduct research and keep learning.

I have a limited amount of life experiences, but if you have questions or think I can be of assistance, please feel free to email me.

News

[October 2026]
Our paper What do Language Models Learn and When? The Implicit Curriculum Hypothesis received an πŸ† Outstanding Paper Award πŸ† at COLM2026.
[September 2026]
How to Interpret Agent Behavior and Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs are accepted into NeurIPS2026. I will be attending NeurIPS in Atlanta, where I am also co-organizing the Queer in AI x {Dis}Ability in AI events. We are also hosting the Interpreting Agent Behavior (IAB) workshop at NeurIPS in Sydney - come say hi!
[September 2026]
New papers πŸ“„: PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research and Last Translation Benchmark are now available on arXiv.
[Summer 2026]
I spent the summer interning at Together AI, working with Fede Bianchi and Kaitlyn Zhou.
Older News
[May 2026]
New paper πŸ“„: How to Interpret Agent Behavior is now available on arXiv.
[April 2026]
New paper πŸ“„: What do Language Models Learn and When? The Implicit Curriculum Hypothesis is now available on arXiv.
[March 2026]
New paper πŸ“„: Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs is now available on arXiv.
[June 2025]
New paper πŸ“„: Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict is now available on arXiv.
[May 2025]
New paper πŸ“„: On the Failure of Latent State Persistence in Large Language Models is now available on arXiv.
[April 2025]
New papers πŸ“„: Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models is accepted into RepL4NLP2025; SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models is accepted into NAACL2025.
[August 2024]
Our system achieve great performance πŸ“ˆ in almost all languages in the IWSLT low-resource speech translation shared task.
[December 2023]
Our paper, The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarks, received an Honorable Mention πŸ† in CoNLL2023.
[October 2023]
Our work Tokenization Consistency Matters for Generative Models on Extractive Tasks is accepted into Findings of EMNLP2023.
[July 2023]
Our work A Replication Study of Compositional Generalization Works on Semantic Parsing was recognized as an πŸ† Outstanding Paper πŸ† in the ML Reproducibility challenge.
[August 2022]
I finished my master's degree, moved to New York City, and will spend a year here for my residency at Meta AI, working with Adina Williams and Dieuwke Hupkes.
[June 2022]
I moved to Santa Clara for my internship at AWS AI, working with Peng Qi and Yuhao Zhang.
[December 2021]
I began as a teaching assistant in Natural Language Processing at UW for winter and spring quarter.
[September 2021]
I began as a teaching assistant in Machine Learning at UW for fall quarter.
[July 2021]
I began my software engineering internship in AuthService team at Amazon.
[May 2021]
Our paper Effective Attention Sheds Light On Interpretability was accepted into Findings of ACL2021. Big thanks to Ana! 🌻
[March 2021]
Finished my Bachelor's degrees πŸ€“ - B.S. in CS and B.A. in math, as well as a minor in classical studies; I began as a teaching assistant in Enterprise Chatbots at UW for spring quarter.
[November 2020]
Joined Noonum as a data scientist intern.
[July 2020]
I began my software engineering internship in AuthEngine team at Amazon.
[September 2019]
Began a project on dual random-walk systems with Professor Christopher Hoffman at Washington Experimental Mathematics Lab.
[July 2019]
Began my internship at National Oceanic & Atmospheric Administration (NOAA) mentored by Dr. Jason Cope.


Cat Warning

I have two very cute cats: Bert (white collar with white paws) and Roberta (grey and looks like a little leopard) - yes, they were adopted in 2019. While you are browsing my website, I hope pictures of Bert and Roberta can give you a relaxing pause.

Bert and Roberta snuggling Bert and Roberta


Misc

My ErdΕ‘s number is 4.