Publications

Also on Google Scholar.

Ordered by authorship position — first author, then co-first author, then other positions. * denotes equal contribution. Detailed write-ups of the main lines of work are on the Research page.

Under Submission

Last Layer Key-Value Cache is All You Need

Adobe Research · under submission
Details

Universal Test-Time Training

Adobe Research · under submission
Details

Test-Time Generation

Adobe Research · under submission
Details

First Author

R-KV: Redundancy-aware KV Cache Compression for Training-Free Reasoning Models Acceleration

Zefan Cai, Wen Xiao, Hanshi Sun, Cheng Luo, Yikai Zhang, Ke Wan, Yucheng Li, Yeyang Zhou, Li-Wen Chang, Jiuxiang Gu, Zhen Dong, Anima Anandkumar, Abedelkadir Asi, Junjie Hu
NeurIPS 2025
PDF · Page · Code · Video · Twitter · Details

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling

Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, Wen Xiao
COLM 2025 Spotlight (24 / 418)
PDF · Page · Code · Twitter · Details

From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models

Zefan Cai*, Haoyi Qiu*, Haozhe Zhao*, Ke Wan, Jiachen Li, Jiuxiang Gu, Wen Xiao, Nanyun Peng, Junjie Hu
TMLR 2025
PDF · Code

Improving Event Definition Following For Zero-Shot Event Detection

Zefan Cai*, Po-Nien Kung*, Ashima Suvarna, Mingyu Derek Ma, Hritik Bansal, Baobao Chang, P. Jeffrey Brantingham, Wei Wang, Nanyun Peng
ACL 2024 (Main) — Long Paper
PDF

DialogVCS: Robust Natural Language Understanding in Dialogue System Upgrade

Zefan Cai*, Xin Zheng*, Tianyu Liu*, Xu Wang, Haoran Meng, Jiaqi Han, Gang Yuan, Binghuai Lin, Baobao Chang, Yunbo Cao
NAACL 2024 (Main) — Long Paper
PDF

MMGR: Multi-Modal Generative Reasoning

Zefan Cai, Haoyi Qiu, Tianyi Ma, Haozhe Zhao, Gengze Zhou, Kung-Hsiang Huang, Parisa Kordjamshidi, Minjia Zhang, Wen Xiao, Jiuxiang Gu, Nanyun Peng, Junjie Hu
Preprint 2025
PDF

Co-First Author

MENTOR: Efficient Autoregressive Image Generation with Balanced Multimodal Control

Haozhe Zhao*, Zefan Cai*, Shuzheng Si, Liang Chen, Jiuxiang Gu, Wen Xiao, Minjia Zhang, Junjie Hu
Findings of ACL 2026
PDF

Test-Time Training with Next-Token Prediction

Xuan Ouyang*, Zefan Cai*, Junjie Hu
Preprint 2026
PDF · Details

Delta Attention Residuals

Cheng Luo*, Zefan Cai*, Junjie Hu
Preprint 2026
PDF · Details

MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning

Haozhe Zhao*, Zefan Cai*, Shuzheng Si*, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, Baobao Chang
ICLR 2024 (Poster)
PDF · Code

Mitigating Language-Level Performance Disparity in mPLMs via Teacher Language Selection and Cross-lingual Self-Distillation

Haozhe Zhao*, Zefan Cai*, Shuzheng Si, Liang Chen, Yufeng He, Kaikai An, Baobao Chang
NAACL 2024 (Main) — Long Paper
PDF · Code

VeCAF: Vision-language Collaborative Active Finetuning with Training Objective Awareness

Rongyu Zhang*, Zefan Cai*, Huanrui Yang*, Zidong Liu, Denis A. Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, Baobao Chang, Yuan Du, Li Du, Shanghang Zhang
ACM Multimedia 2024
PDF

Compositional Task Representations for Large Language Models

Nan Shao*, Zefan Cai*, Chonghua Liao, Yanan Zheng, Zhilin Yang
ICLR 2023 (Poster)
PDF · Code

SANTA: Separate Strategies for Inaccurate and Incomplete Annotation Noise in Distantly-Supervised Named Entity Recognition

Shuzheng Si*, Zefan Cai*, Shuang Zeng, Guoqiang Feng, Jiaxing Lin, Baobao Chang
ACL 2023 (Findings) — Long Paper
PDF

ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks

Yuliang Liu*, Xiangru Tang*, Zefan Cai*, Junjie Lu, Yichi Zhang, Yanjun Shao, Zexuan Deng, Helan Hu, Zengxian Yang, Kaikai An, Ruijun Huang, Shuzheng Si, Sheng Chen, Haozhe Zhao, Zhengliang Li, Liang Chen, Yiming Zong, Yan Wang, Tianyu Liu, Zhiwei Jiang, Baobao Chang, Yujia Qin, Wangchunshu Zhou, Yilun Zhao, Arman Cohan, Mark Gerstein
ICLR 2025 DL4C Workshop & Agentic AI for Scientific Discovery Workshop
PDF · Page · Code · Data

Other Positions

LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback

Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Junjie Hu, Tianyu Liu, Baobao Chang
ACL 2025 (Findings) — Long Paper
PDF · Code

Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning

Yu Fu, Zefan Cai, Abedelkadir Asi, Wayne Xiong, Yue Dong, Wen Xiao
ICLR 2025 (Poster)
PDF · Code · Details

No Preference Left Behind: Group Distributional Preference Optimization

Binwei Yao, Zefan Cai, Yun-Shiuan Chuang, Shanglin Yang, Ming Jiang, Diyi Yang, Junjie Hu
ICLR 2025 (Poster)
PDF · Code

HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading

Cheng Luo, Zefan Cai, Hanshi Sun, Jinqi Xiao, Bo Yuan, Wen Xiao, Junjie Hu, Jiawei Zhao, Beidi Chen, Anima Anandkumar
ICML 2025 Workshop on Long-Context Foundation Models
PDF · Code · Details

Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigm

Lin Zhang, Zefan Cai, Yufan Zhou, Shentong Mo, Jinhong Lin, Cheng-En Wu, Yibing Wei, Yijing Zhang, Ruiyi Zhang, Wen Xiao, Tong Sun, Junjie Hu, Pedro Morgado
Preprint 2025
PDF

A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Fine-grained Image Generation

Liang Chen, Sinan Tan, Zefan Cai, Weichu Xie, Haozhe Zhao, Yichi Zhang, Junyang Lin, Jinze Bai, Tianyu Liu, Baobao Chang
ICLR 2025 (Poster)
PDF · Code · Model

UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation

Lin Zhang, Sicheng Mo, Zefan Cai, Jinhong Lin, Zihao Lin, Jiuxiang Gu, Krishna Kumar Singh, Yuheng Li, Yin Li
Preprint 2026
PDF

Omni-Math: A Universal Olympiad Level Mathematic Benchmark for Large Language Models

Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, Baobao Chang
ICLR 2025 (Poster)
PDF · Page · Code · Data

COMMA: A Communicative Multimodal Multi-Agent Benchmark

Timothy Ossowski, Jixuan Chen, Danyal Maqbool, Zefan Cai, Tyler Bradshaw, Junjie Hu
TMLR 2025
PDF

Large Language Models Are Not Fair Evaluators

Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, Zhifang Sui
ACL 2024 (Main) — Long Paper
PDF · Code

PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chain

Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Xiangdi Meng, Tianyu Liu, Baobao Chang
ACL 2024 (Findings) — Long Paper
PDF · Code · Data

Towards End-to-End Embodied Decision Making via Multi-Modal Large Language Model

Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Tianyu Liu, Baobao Chang
NeurIPS 2023 Foundation Models for Decision Making Workshop
PDF · Code

Improving the Robustness of Distantly-Supervised Named Entity Recognition via Uncertainty-Aware Teacher Learning and Student-Student Collaborative Learning

Shuzheng Si, Helan Hu, Haozhe Zhao, Shuang Zeng, Kaikai An, Zefan Cai, Baobao Chang
ACL 2024 (Findings) — Long Paper
PDF

AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence

Yuliang Liu, Junjie Lu, Zhaoling Chen, Chaofeng Qu, Jason Klein Liu, Chonghan Liu, Zefan Cai, Yunhui Xia, Li Zhao, Jiang Bian, Chuheng Zhang, Wei Shen, Zhouhan Lin
ICML 2025 (Poster)
PDF · Code · Model

BabyVision: Visual Reasoning Beyond Language

Liang Chen, Weichu Xie, Yiyan Liang, Hongfeng He, Hans Zhao, Zhibo Yang, Zhiqi Huang, Haoning Wu, Haoyu Lu, Y. Charles, Yiping Bao, Yuantao Fan, Guopeng Li, Haiyang Shen, Xuanzhong Chen, Wendong Xu, Shuzheng Si, Zefan Cai, Wenhao Chai, Ziqi Huang, Fangfu Liu, Tianyu Liu, Baobao Chang, Ming Wu, Xiaobo Hu, Kaiyuan Chen, Yixin Ren, Yang Liu, Yuan Gong, Kuan Li
Preprint 2026
PDF