Publications

Papers
Debangshu Banerjee, Changming Xu, Eugene Ie, Ming Zhang, Daiyi Peng, Chu-Cheng Lin, Gagandeep Singh
arXiv 2026
Combines formal specifications with gradient-based learning so self-evolving LLM agents improve while provably meeting their contracts.
Chu-Cheng Lin, Daiyi Peng, Yifeng Lu, Ming Zhang, Eugene Ie
arXiv 2025
Treats LLM workflows as typed probabilistic programs with parameter-efficient adapters, so the whole workflow can be trained end-to-end with gradients instead of prompt tuning.
Gemini Team, Google (including Daiyi Peng)
Technical report, 2025
The Gemini 2.X model family, with state-of-the-art coding and reasoning, long video understanding and new agentic workflows.
Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, Quoc V. Le
NeurIPS 2024
Introduces the LongFact benchmark, SAFE (a search-augmented automatic fact checker) and the F1@K metric for measuring factuality in long answers.
Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, Andrew M. Dai
arXiv 2024
A survey of how synthetic data is generated and used to train and evaluate language models, and how to keep it factual, faithful and unbiased.
Chongyang Gao, Kezhen Chen, Jinmeng Rao, Baochen Sun, Ruibo Liu, Daiyi Peng, Yawen Zhang, Xiaoyuan Guo, Jie Yang, VS Subrahmanian
arXiv 2024
Introduces MoLA, which allocates different numbers of LoRA experts to different Transformer layers, showing that more experts in higher layers gives better results with fewer parameters.
Xingyou Song, Oscar Li, Chansoo Lee, Bangding Yang, Daiyi Peng, Sagi Perel, Yutian Chen
TMLR 2024
Trains language models to do numeric regression from text alone, beating conventional regressors when trained across many tasks.
Gemini Team, Google (including Daiyi Peng)
Technical report, 2023
The first Gemini models, natively multimodal across image, audio, video and text, with Gemini Ultra setting the state of the art on 30 of 32 benchmarks.
Yanqi Zhou, Nan Du, Yanping Huang, Daiyi Peng, Chang Lan, Da Huang, Siamak Shakeri, David So, Andrew M. Dai, Yifeng Lu, Zhifeng Chen, Quoc V. Le, Claire Cui, James Laudon, Jeff Dean
ICML 2023
A non-uniform Transformer architecture found by evolutionary search over layer types and sparsity, training faster and scaling better than hand-designed models like GLaM.
Yicheng Fan, Dana Alon, Jingyue Shen, Daiyi Peng, Keshav Kumar, Yun Long, Xin Wang, Fotis Iliopoulos, Da-Cheng Juan, Erik Vee
arXiv 2023
Searches architectures layer by layer, turning multi-objective NAS into a problem of polynomial complexity while still finding better models.
Daiyi Peng, Xuanyi Dong, Esteban Real, Yifeng Lu, Quoc V. Le
arXiv 2023
Uses symbolic patches so researchers can share ML ideas as rules that apply to models they have never seen, cutting code across teams.
Yingwei Li, Adams Wei Yu, Tianjian Meng, Ben Caine, Jiquan Ngiam, Daiyi Peng, Junyang Shen, Yifeng Lu, Denny Zhou, Quoc V. Le, Alan Yuille, Mingxing Tan
CVPR 2022
Fuses lidar and camera features deep in the network, using InverseAug and LearnableAlign to keep them aligned, for better 3D detection in self-driving.
Yingjie Miao, Xingyou Song, John D. Co-Reyes, Daiyi Peng, Summer Yue, Eugene Brevdo, Aleksandra Faust
AutoML Conference 2022
Shows that DARTS finds better policy networks for reinforcement learning across algorithms and environments, at about 3x the cost of training one model.
John D. Co-Reyes, Yingjie Miao, Daiyi Peng, Esteban Real, Sergey Levine, Quoc V. Le, Honglak Lee, Aleksandra Faust
arXiv 2021
Meta-learns RL algorithms by searching over computational graphs of loss functions, rediscovering temporal-difference learning and finding variants that improve on DQN.
Yanqi Zhou, Xuanyi Dong, Berkin Akin, Mingxing Tan, Daiyi Peng, Tianjian Meng, Amir Yazdanbakhsh, Da Huang, Ravi Narayanaswami, James Laudon
arXiv 2021
Jointly searches neural architectures and edge accelerator configurations, improving ImageNet accuracy by about 1% or cutting energy by up to 2x.
Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Qiuyi Zhang, Daiyi Peng, Deepali Jain, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Yuxiang Yang
arXiv 2021
Combines evolution strategies with combinatorial optimizers to search spaces that mix discrete and continuous parameters.
Yanqi Zhou, Xuanyi Dong, Daiyi Peng, Ethan Zhu, Amir Yazdanbakhsh, Berkin Akin, Mingxing Tan, James Laudon
OpenReview 2021
Searches neural architectures and hardware accelerator designs together rather than separately, for better accuracy and latency on edge devices.
Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, Quoc V. Le
arXiv 2020
A single pipeline that searches architectures and hyperparameters together, alternating weight training with an RL controller.
Daniel S. Park, Jaehoon Lee, Daiyi Peng, Yuan Cao, Jascha Sohl-Dickstein
arXiv 2020
Uses Neural Network Gaussian Process inference as a cheap signal for ranking architectures during neural architecture search.
Daiyi Peng, Xuanyi Dong, Esteban Real, Mingxing Tan, Yifeng Lu, Gabriel Bender, Hanxiao Liu, Adam Kraft, Chen Liang, Quoc V. Le
NeurIPS 2020 (oral)
A symbolic programming library that decouples search spaces, search algorithms and child programs, so AutoML ideas can be changed without rewriting program logic.
Jiquan Ngiam, Daiyi Peng, Vijay Vasudevan, Simon Kornblith, Quoc V. Le, Ruoming Pang
arXiv 2018
Shows that choosing pre-training data matters for transfer, and reweights it by importance to the target domain to improve fine-grained classification.
Patents
Daiyi Peng, Yifeng Lu, Quoc V. Le
US Patent 12,443,882 B2, 2025 (continuation US 2026/0105369 A1)
Yicheng Fan, Keshav Kumar, Andrew Tomkins, Dana Alon, Erik Nathan Vee, Shanmugasundaram Ravikumar, Jingyue Shen, Daiyi Peng
US 2025/0238683 A1 (application), 2025
Lav Rai, Xiang Xu, Yen-Min Hsu, Bo Wu, Daiyi Peng
US 2024/0289605 A1 (application), 2024
Yanqi Zhou, Yanping Huang, Yifeng Lu, Andrew M. Dai, Siamak Shakeri, Zhifeng Chen, James Laudon, Quoc V. Le, Da Huang, Nan Du, David Richard So, Daiyi Peng, Yingwei Cui, Jeffrey Adgate Dean, Chang Lan
US 2024/0112027 A1 (application), 2024
Yanqi Zhou, Amir Yazdanbakhsh, Berkin Akin, Daiyi Peng, Yuxiong Zhu, Mingxing Tan, Xuanyi Dong
US 2024/0005129 A1 (application), 2024
John Dalton Co-Reyes, Yingjie Miao, Daiyi Peng, Sergey Vladimir Levine, Quoc V. Le, Honglak Lee, Aleksandra Faust
US 2022/0391687 A1 (application), 2022
Jaehoon Lee, Daiyi Peng, Yuan Cao, Jascha Narain Sohl-Dickstein, Daniel Sung-Joon Park
US 2022/0019856 A1 (application), 2022
Mingxing Tan, Xuanyi Dong, Wei Yu, Quoc V. Le, Daiyi Peng
US 2021/0383223 A1 (application), 2021
Vijay Vasudevan, Ruoming Pang, Quoc V. Le, Daiyi Peng, Jiquan Ngiam, Simon Kornblith
US 2020/0104710 A1 (application), 2020