DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning(ECCV-2022 Oral)

This repository contains the Official Pytorch Implementation for DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

@article{gao2021disco,
  title={DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning},
  author={Yuting Gao, Jia-Xin Zhuang, Shaohui Lin, Hao Cheng, Xing Sun, Ke Li, Chunhua Shen},
  journal={European Conference on Computer Vision(ECCV)},
  year={2022}
}

If the project is useful to you, please give us a star. ⭐️

Framework

Checkpoints

Teacher Models

For teacher models such as ResNet-50*2 etc, we use their official implementation, which can be downloaded from their github pages.

Student Models by DisCo

Requirements

• Python3

• Pytorch 1.6+

• Detectron2

• 8 GPUs are preferred

• ImageNet, Cifar10/100, VOC, COCO

Reproduction

Commands can be found on Reproduction.

Thanks

Code heavily depends on MoCo-V2, Detectron2.