Continual Learning

I was the tech lead for Continual Learning at Scale at Facebook AI. We built state-of-the-art continual learning methods and a platform that let production models keep learning from fresh data without forgetting what they already knew.

Research Projects

CLAP: Continuous Learning as a Platform

CLAP is a modular platform built on top of PyTorch that brings state-of-the-art continual learning techniques to production models:

  • Continuous samplers: importance-based and cluster-based sampling.
  • Prompt-based continual learning.
  • Memory rehearsal techniques.
  • Self-supervision, so models adapt to real-time streaming data instead of waiting a week or more for annotations.
Overview of the continual learning platform
Overview of continual learning at scale.

The trade-offs we optimized

Forgetting vs. intransigence

Adapt quickly to new data without erasing previously learned knowledge.

Performance vs. back-compatibility

Improve models continuously while staying compatible with the systems that consume them.

Accuracy vs. performance

Get the accuracy gains of continual updates within production cost and latency budgets.

Impact

CLAP was a horizontal technology that improved performance across diverse use cases, with 3–10x improvement in recall at the operating precisions: integrity violation detection, media and creator ranking in Instagram, COVID and vaccine misinformation, and election misinformation.

Multimodal self-supervision: Refiner Fusion Networks

Refiner Fusion Networks refine multimodal representations with a modality-centric self-supervised module, one of the building blocks for learning from unlabeled streaming data.

Related

Computer Vision: Automatic Alt Text at Facebook

Selected Publications

  1. Sankaran, S., Yang, D. and Lim, S.-N. Multimodal Fusion Refiner Networks. arXiv:2104.03435, 2021. [arXiv]
  2. Sankaran, S., Yang, D. and Lim, S.N. Refining multimodal representations using a modality-centric self-supervised module, 2022. [OpenReview]