Ming Li

minglii [AT] umd.edu

profile.png

I am a second-year Ph.D. student in Computer Science at the University of Maryland, advised by Prof. Tianyi Zhou. I began my academic journey in computer science with a Bachelor of Science from Xi’an Jiaotong University in 2020, followed by a Master of Science at Texas A&M University advised by Prof. Ruihong Huang in 2023. Besides, I spent 2 years at Shenzhen Institutes of Advanced Technology, Chinese Academy of Science, advised by Prof. Yu Qiao since 2019.

My research interests broadly lie in the areas of Machine Learning (ML), Natural Language Processing (NLP), and Large Language Models (LLM). More specifically, my recent research interests mainly lie in Instruction Tuning for LLMs, including: (i) Data Selection (Cherry LLM (IFD), Superfiltering); (ii) Data Synthesis (Reflection-Tuning, Selective Reflection-Tuning); (iii) Data Augmentation (Mosaic-IT, RuleR); (iv) Controllability (DEBATunE, RuleR); (iv) Interpretability (Layer_Gradient). I am also interested in and now exploring Vision-LLMs Finetuning, RAG, and Agent.

If you are looking for a highly motivated intern with a background in computer science and a passion for advancing AI technologies, I would be thrilled to have an opportunity to chat with you!

I am really interested in collaboration, feel free to drop me an email for any opportunity!

news

Nov 10, 2024 I will be attending EMNLP2024 in Miami from Nov.11 to Nov.14, feel free to have a chat with me!!
Oct 31, 2024 One paper was put on the arXiv: What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective, where we try to understand the layer-wise gradient behaviors when LLMs are finetuned on Fast vs. Slow Thinking. Repo: Layer_Gradient.
Jun 22, 2024 One paper was put on the arXiv: RuleR: Improving LLM Controllability by Rule-based Data Recycling, where we proposed an augmentation method that incorporates multiple rule-based constraints into the original instruction data. Repo: RuleR.
May 23, 2024 One paper was put on the arXiv: Mosaic IT: Enhancing Instruction Tuning with Data Mosaics, where we proposed an augmentation method for instruction tuning, which concurrently improves the LLM performances and lowers the training expenses. Repo: Mosaic-IT.
May 16, 2024 Three papers were accepted by ACL 2024!
1 Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning;
2 Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning;
3 Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements (DEBATunE).
Mar 13, 2024 One paper was accepted by NAACL 2024!
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning (Cherry LLM (IFD))
Feb 21, 2024 I will join Adobe (based in San Jose) as a Research Scientist/Engineer Intern this Summer~
Feb 20, 2024 One Survey was put on the arXiv: A Survey on Knowledge Distillation of Large Language Models. Repo: Awesome-Knowledge-Distillation-of-LLMs.
Oct 28, 2023 One paper was accepted by Instruction Workshop @ NeurIPS 2023!
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning.
Oct 07, 2023 One paper was accepted by EMNLP 2023!
PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adapter.
Sep 01, 2023 I arrived at the University of Maryland, officially beginning my journey for a Ph.D. ✌️
Jun 01, 2023 I obtained my Master’s in Computer Science at Texas A&M University.

selected publications

  1. ACL
    Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
    Ming Li ,  Yong Zhang ,  Shwai He , and 5 more authors
    2024
  2. ACL
    Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
    Ming Li ,  Lichang Chen ,  Jiuhai Chen , and 3 more authors
    2024
  3. ACL
    Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
    Ming Li ,  Jiuhai Chen ,  Lichang Chen , and 1 more author
    2024
  4. NAACL
    From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
    Ming Li ,  Yong Zhang ,  Zhitao Li , and 6 more authors
    2024
  5. NIPS Workshop
    Reflection-Tuning: Recycling Data for Better Instruction-Tuning
    Ming Li ,  Lichang Chen ,  Jiuhai Chen , and 2 more authors
    In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following , 2023