PhD Research Intern, Visual Object Understanding

About Our Internship Program
 
Zoox’s internship program offers hands-on experience with cutting-edge technology, mentorship from some of the industry’s brightest minds, and the opportunity to make meaningful contributions to real projects. We seek interns who demonstrate strong academic performance, engagement beyond the classroom, intellectual curiosity, and a genuine interest in Zoox’s mission.
 
Project Overview
 
The Perception Attributes team builds the agent semantics layer of Zoox's perception stack. Our models classify what obstacles mean — detecting emergency vehicle lights, pedestrian gestures, turn signals, and dozens of other behavioral signals that inform how the AV responds. This work sits at the intersection of safety-critical autonomy and cutting-edge ML: our models run on every Zoox vehicle, and our outputs directly influence decisions like yielding to emergency vehicles and interacting with construction workers. The team is small, moves fast, and collaborates closely with ML researchers across the AI org. 
 
During the internship, you will work on one of the most exciting open problems in AV perception: using modern foundation models — large vision-language models, multimodal transformers, and audio-visual architectures — to dramatically expand the semantic understanding of our perception stack. Current approaches require months of data collection and labeling to add a single new attribute class. The research goal is to change that fundamentally, using VLMs and language-aligned representations to make our models more generalizable, queryable, and data-efficient. The work spans dataset construction, model design, and evaluation — with direct implications for how Zoox handles novel emergency vehicles, complex pedestrian behavior, and safety-critical edge cases as we scale to new cities.