David R. So

I am a quantitative researcher working at Jane Street Capital with a focus on machine learning.

Previously, I was a staff researcher working at Google Brain/DeepMind. My published work focused on AutoML, NLP and LLMs. I was also a technical lead and core contributor for Google's largest language modeling efforts, including Bard, PaLM 2, LaMDA 2, and Meena.

Publications

PaLM 2 Technical Report

Google (Core Contributor)

Brainformers: Trading Simplicity for Efficiency

Yanqi Zhou, Nan Du, Yanping Huang, Daiyi Peng, Chang Lan, Da Huang, Siamak Shakeri,
David R. So, Andrew M. Dai, Yifeng Lu, Zhifeng Chen, Quoc V. Le, Claire Cui, James Laudon, Jeff Dean

In International Conference on Machine Learning 2023

EvoPrompting: Language Models for Code-Level Neural Architecture Search

Angelica Chen, David M. Dohan, David R. So

Unified Functional Hashing in Automatic Machine Learning

Ryan Gillard, Stephen Jonany, Yingjie Miao, Michael Munn, Connal de Souza, Jonathan Dungay, Chen Liang, David R. So, Quoc V. Le, Esteban Real

Transcending Scaling Laws with 0.1% Extra Compute

Yi Tay, Jason Wei, Hyung Won Chung, Vinh Q Tran, David R. So, Siamak Shakeri, Xavier Garcia, Huaixiu Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V Le, Mostafa Dehghani

The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink

David Patterson, Joseph Gonzalez, Urs Hölzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R. So, Maud Texier, Jeff Dean

In Computer 2022

Primer: Searching for Efficient Transformers for Language Modeling

David R. So, Wojciech Mańke, Hanxiao Liu, Zihang Dai, Noam Shazeer, Quoc V. Le

In Conference on Neural Information Processing Systems 2021

Pay Attention to MLPs

Hanxiao Liu, Zihang Dai, David R. So, Quoc V. Le

In Conference on Neural Information Processing Systems 2021

Carbon Emissions and Large Neural Network Training

David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang,

Lluis-Miquel Munguia, Daniel Rothchild, David R. So, Maud Texier, Jeff Dean

MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records

Zhen Xu*, David R. So*, Andrew M. Dai

In Association for the Advancement of Artificial Intelligence 2021

AutoML-Zero: Evolving Machine Learning Algorithms from Scratch

Esteban Real, Chen Liang, David R. So, Quoc Le

In International Conference on Machine Learning 2020

Towards a Human-Like Open-Domain Chatbot

Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel,

Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu,

Quoc V. Le

The Evolved Transformer

David R. So, Quoc V. Le, Chen Liang

In International Conference on Machine Learning 2019

Evolving Modular Neural Sequence Architectures with Genetic Programming

David Dohan, David R. So, Quoc V. Le

In Proceedings of the Genetic and Evolutionary Computation Conference Companion 2018

Classification of Crystallization Outcomes Using Deep Convolutional Neural Networks

Andrew E Bruno, Patrick Charbonneau, Janet Newman, Edward H Snell, David R. So, Vincent Vanhoucke, Christopher J Watkins, Shawn Williams, Julie Wilson

In PLOS One 2018

Improving Image Generative Models with Human Interactions

Andrew K. Lampinen, David R. So, Douglas Eck, Fred Bertsch