Abstract:
In some domains like autonomous driving, we can collect millions of hours of expert demonstrations yet lack any notion of which moments are truly worth learning from; in others, like long-term point tracking, obtaining real annotated data at scale is nearly impossible. In this talk, I cover two approaches that address these opposite bottlenecks.
Verifier enables real-world fine-tuning of point trackers by estimating the reliability of tracker predictions and selecting trustworthy pseudo-labels from unlabeled videos, substantially improving sim-to-real performance. RELACS, trained on our counterfactual driving dataset COCA, learns to score driving behavior directly from video without privileged information, generalizing from synthetic rollouts to real-world driving and proving useful for curating large datasets. Together, these works demonstrate how models can
help navigate both extremes by determining how to filter it when data is abundant and generating supervision when real-world, labeled data does not exist at scale.
Biography:
Fatma Güney is an Assistant Professor at Koc University in Istanbul. She received her PhD from the Max Planck Institute in Germany. Her research focuses on computer vision problems related to autonomous driving. She is a recipient of the ERC Starting Grant as well as prestigious fellowships including the Newton Fund Advanced Fellowship and the Marie Curie Individual Fellowship. She regularly serves as a reviewer with multiple outstanding reviewer awards in top-tier AI conferences.