Code for our paper: Label, Verify, Correct: A Simple Few-Shot Object Detection Method
Abstract: The objective of this paper is few-shot object detection (FSOD) – the task of expanding an object detector for a new category given only a few instances for training. We introduce a simple pseudo-labelling method to source high-quality pseudo-annotations from the training set, for each new category, vastly increasing the number of training instances and reducing class imbalance; our method finds previously unlabelled instances. Naïvely training with model predictions yields sub-optimal performance; we present two novel methods to improve the precision of the pseudo-labelling process: first, we introduce a verification technique to remove candidate detections with incorrect class labels; second, we train a specialised model to correct poor quality bounding boxes. After these two novel steps, we obtain a large set of high-quality pseudo-annotations that allow our final detector to be trained end-to-end. Additionally, we demonstrate our method maintains base class performance, and the utility of simple augmentations in FSOD. While benchmarking on PASCAL VOC and MS-COCO, our method achieves state-of-the-art or second-best performance compared to existing approaches across all number of shots.
If you find this repository useful for your own research, please consider citing our paper.
@InProceedings{Kaul22,
author = "Prannay Kaul and Weidi Xie and Andrew Zisserman",
title = "Label, Verify, Correct: A Simple Few-Shot Object Detection Method",
booktitle = "IEEE Conference on Computer Vision and Pattern Recognition",
year = "2022",
}
This repo began from the excellent Fsdet repo.
Requirements
Build lvc We recommend using conda to create an enviroment for this project
conda create --name lvc
conda activate lvc
Install PyTorch. You can choose the PyTorch and CUDA version according to your machine. Just make sure your PyTorch version matches the prebuilt Detectron2 version (next step). For example for PyTorch v1.7.1:
pip install torch==1.7.1 torchvision==0.8.2
This code uses Detectron2 v0.2.1, however for ease we have modified this code and a self-contained version exists in this repo.
To install our Detectron2 code use pip locally, ensuring our detectron2/
directory is in the present working directory
python -m pip install -e .
The installation of our Detectron2 code is identical to the official Detectron2 code, so if you run into errors at the above step please see the installation docs for Detectron2, specifically the common issues.
Install all other dependencies
python -m pip install -r requirements.txt
We evaluate our models on three datasets:
See datasets/README.md for more details.
LVC is conceptually simple but the full run involves many commands. For a full explanation of how to train for LVC, see TRAIN_FULL.md.