Zekun Li’s Homepage

My name in Chinese: 李泽坤

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/cd361d2a-9856-49d3-bfa3-abc0d900c2fa/25231.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/cd361d2a-9856-49d3-bfa3-abc0d900c2fa/25231.png" width="40px" /> Github

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/f30b1dd3-7384-4ae2-b912-11f36f7e174f/Picture2.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/f30b1dd3-7384-4ae2-b912-11f36f7e174f/Picture2.png" width="40px" /> Google Scholar

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/f3ef8b9f-d4d4-4dd3-8fe4-b19b251c7eec/Picture1.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/f3ef8b9f-d4d4-4dd3-8fe4-b19b251c7eec/Picture1.png" width="40px" /> Semantic Scholar

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/4a681ff9-9560-425e-a09b-83cefa5ab4e8/twitter-3.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/a91125d9-b31c-4ba3-8d0b-3f5f74df148f/4a681ff9-9560-425e-a09b-83cefa5ab4e8/twitter-3.png" width="40px" /> Twitter

</aside>

<aside> 🖇️

</aside>

👋 Hi!

I am Zekun Li, a forth-year Ph.D. candidate in the Computer Science Department at the University of California, Santa Barbara (UCSB), advised by Professors William Wang and Xifeng Yan at UCSB NLP Group. My research primarily focuses on developing controllable language model agents for conversation, reasoning, planning, and tool use in complex tasks and scenarios.

I have interned at Microsoft Research (2023), Meta Reality Labs (2023), and Google Deepmind Gemini Team (2024). I am honored to have been selected as a recipient of the 2024 J.P. Morgan PhD Fellowship Award (around 10 recipients each year).

I am actively looking for full time industry positions! Feel free to drop me an email if you're interested!

🌋 Research Interests

My research interests lie in natural language processing and machine learning. Specifically:

Language Agents: He studies controllable large language model-based agents capable of conversation, instruction-following, reasoning, and tool use to address complex tasks and scenarios, with a specific focus on alignment, robustness, and safety. I have gained substantial experience in developing real-world conversational agents and chatbots through academic research, participation in the Alexa Prize Challenges, and relevant internships in leading companies like Microsoft Research and Meta Reality Labs in 2023 and Google Gemini Deepmind in 2024.
Multimodal Study: I am also interested in multimodal study, encompassing language, vision, and time series, for solving challenges in scientific and professional fields.
Recommender Systems: Prior to PhD study, he focused on recommender systems, particularly click-through rate (CTR) prediction, cold-start recommendation, and outfit recommendation. My work FiGNN is the first graph-based approach and leading solution in CTR prediction.

📜 Publications

(*: equal contribution)

Language Model Agents:

"Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection."

Zekun Li, Baolin Peng, Pengcheng He, Xifeng Yan

EMNLP 2024 Main paper

"MMSci: A Multimodal Multi-Discipline Dataset for PhD-Level Scientific Comprehension."

Zekun Li, Xianjun Yang, Kyuri Choi, Wanrong Zhu, Ryan Hsieh, HyeonJung Kim, Jin Hyuk Lim, Sungyoung Ji, Byungju Lee, Xifeng Yan, Linda Ruth Petzold, Stephen D. Wilson, Woosang Lim, William Yang Wang

AI4MAT-Vienna 2024 (Spotlights) paper code & data

"Can Editing LLMs Inject Harm?"

Canyu Chen*, Baixiang Huang*, Zekun Li, Zhaorun Chen, Shiyang Lai, Xiongxiao Xu, Jia-Chen Gu, Jindong Gu, Huaxiu Yao, Chaowei Xiao, Xifeng Yan, William Yang Wang, Philip Torr, Dawn Song, Kai Shu

TiFA@ICML 2024 paper website code & data

"Large Language Models as Zero-shot Dialogue State Tracker through Function Calling."

Zekun Li, Zhiyu Chen, Mike Ross, Patrick Huber, Seungwhan Moon, Zhaojiang Lin, Luna Dong, Adithya Sagar, Xifeng Yan, Paul A. Crook

ACL 2024 Main paper code

"Guiding Large Language Models via Directional Stimulus Prompting."

Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, Xifeng Yan

NeurIPS 2023 paper code website media coverage

"Explanations from Large Language Models Make Small Reasoners Better."

Shiyang Li, Jianshu Chen, Yelong Shen, Zhiyu Chen, Xinlu Zhang, Zekun Li, Hong Wang, Jing Qian, Baolin Peng, Yi Mao, Wenhu Chen, Xifeng Yan

SAI@AAAI 2024 paper

"ChatEdit: Towards Multi-turn Interactive Facial Image Editing via Dialogue."

Xing Cui*, Zekun Li*, Peipei Li, Yibo Hu, Hailin Shi, Zhaofeng He

EMNLP 2023 Main paper

"Limitations of Language Models in Arithmetic and Symbolic Induction."

Jing Qian*, Hong Wang*, Zekun Li, Shiyang Li, Xifeng Yan

ACL 2023 Main paper

"Controllable Dialogue Simulation with In-Context Learning."

Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing Qian, Xifeng Yan

EMNLP 2022 Findings paper code

"Making Something out of Nothing: Building Robust Task-oriented Dialogue Systems from Scratch."

Zekun Li*, Hong Wang*, Alon Albalak, Yingrui Yang, Jing Qian, Shiyang Li, Xifeng Yan

1st Proceedings of Alexa Prize TaskBot (Alexa Prize 2021) paper demo

Multimodal Study:

"Localize, Understand, Collaborate: Semantic-Aware Dragging via Intention Reasoner."

Xing Cui, Peipei Li, Zekun Li, Xuannan Liu, Yueying Zou, Zhaofeng He

NeurIPS 2024 paper

"FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLM."

Xuannan Liu, Peipei Li, Huaibo Huang, Zekun Li, Xing Cui, Jiahao Liang, Lixiong Qin, Weihong Deng, Zhaofeng He

ACM Multimedia (MM) 2024 paper

"InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser."

Xing Cui, Zekun Li, Peipei Li, Huaibo Huang, Zhaofeng He

ECCV 2024 paper code

"Time Series as Images: Vision Transformer for Irregularly Sampled Time Series."

Zekun Li, Shiyang Li, Xifeng Yan

NeurIPS 2023 paper code

Recommender Systems:

"Cold-start Sequential Recommendation via Meta Learner."

Yujia Zheng, Siyi Liu, Zekun Li, Shu Wu

AAAI 2021 paper

"DGTN: Dual-channel Graph Transition Network for Session-based Recommendation."

Yujia Zheng, Siyi Liu, Zekun Li, Shu Wu

NeuRec@ICDM 2020 paper

"Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction."

Zekun Li*, Zeyu Cui*, Shu Wu, Xiaoyu Zhang, Liang Wang

CIKM 2019 paper code

"Dressing as a Whole: Outfit Compatibility Learning Based on Node-wise Graph Neural Networks."

Zeyu Cui*, Zekun Li*, Shu Wu, Xiaoyu Zhang, Liang Wang

WWW 2019 paper code

"NeuroStylist: Neural Compatibility Modeling for Clothing Matching."

Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, Jun Ma

ACM Multimedia (MM) 2017 paper code data

Preprints

"Breaking Mental Set to Improve Reasoning through Diverse Multi-Agent Debate."

Yexiang Liu, Jie Cao, Zekun Li, Ran He, Tieniu Tan

Arxiv, 2024

"MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs."

Xuannan Liu, Zekun Li, Peipei Li, Shuhan Xia, Xing Cui, Linzhi Huang, Huaibo Huang, Weihong Deng, Zhaofeng He

Arxiv, 2024 paper

"AlpaCare: Instruction-tuned Large Language Models for Medical Application."

Xinlu Zhang, Chenxin Tian, Xianjun Yang, Licahng Chen, Zekun Li, Linda Ruth Petzold

Arxiv, 2023 paper code

"Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals."

Zhiyuan Hu, Yue Feng, Yang Deng, Zekun Li, See-Kiong Ng, Anh Tuan Luu, Bryan Hooi

Arxiv, 2023 paper