Jiashuo Liu（刘家硕）

ByteDance Seed Team (prev)
Member of TopSeed Talent Program (prev)

Email: liujiashuo77@gmail.com

ScholarTwitterGithubCV
Jiashuo Liu

I work on LLM Eval & Synthetic Data.

I was a Research Scientist at ByteDance Seed, as a member of TopSeed Program, working with Wenhao Huang. I received my Ph.D. from the Department of Computer Science and Technology at Tsinghua University, advised by Professors Peng Cui and Bo Li. Previously, I was a visiting student researcher at Stanford University, working with Professor Jose Blanchet; and at the Cambridge Centre for AI in Medicine, working with Professor Mihaela van der Schaar.I obtained my Bachelor's degree from Tsinghua University.

Research

I focus on building next-generation LLM evaluation systems and large-scale, data-centric AI. I believe LLM evaluation should be reliable, efficient, and anchored in realistic real-world tasks, and it must rest on a deep understanding of data. I contribute to the Seed Evaluation System at ByteDance Seed (part of my insights are demonstrated in Seed1.8 Model Card).

Specific topics include:
  1. Reliable LLM Eval: We design new perturbation algorithms and future prediction tasks with completely no data contamination to reliably assess the true capabilities of Agentic LLMs. Together with Prof. Mengdi Wang, our FutureX project,described by Elon Musk as the "Best Measure of Intelligence"(media), has received over 100 million views on X(Twitter). Following this line, we release CryptoBench for expert-level tasks in cryptocurrency domains. Also, we design LLM Swiss Round to give a holistic ranking of leading LLMs.
  2. Efficient LLM Eval: We design algorithms to efficiently and accurately evaluate LLM's performance.
  3. Realistic LLM Eval: We use realistic, professional and valuable tasks to benchmark LLM's true value in helping with daily works. Together with Prof. Hongseok Namkoong, our FinSearchComp has received over 20 million views on X(Twitter), and is recently adopted in MiniMax-M2's report and Kimi-K2-Thinking's report.
  4. LLM Calibration: We design Behaviorally Calibrated-RL to calibrate LLM's confidence estimation, enabling LLM's to say "I don't know".
Current Academic Collaborators:
Jose Blanchet (Stanford MS&E), Emmanuel Candès (Stanford Statistics), Hongseok Namkoong (Columbia DRO), Mihaela van der Schaar (Cambridge CCAIM), and Mengdi Wang (Princeton ECE).
Current Interns:
Zerui Cheng (Princeton University, reliable evaluation), Jiayun Wu (CMU, LLM calibration)

Education/Experience

  • 2020-2025: Ph.D. in Computer Science, Tsinghua University
  • 2016-2020: B.Eng in Computer Science, Tsinghua University

  • 2024/6-2024/11: Visiting Student Researcher at the University of Cambridge
  • 2023/10-2024/3: Visiting Student Researcher at Stanford University

Template adapted from Danqi Chen's.