[📖 Paper] [🤗 Huggingface Model]
In this paper, we present a novel collaborative hybrid assignments training scheme, namely Co-DETR, to learn more efficient and effective DETR-based detectors from versatile label assignment manners.
Model | Backbone | Aug | Dataset | box AP (val) | mask AP (val) | box AP (test) | mask AP (test) | Config | Download |
---|---|---|---|---|---|---|---|---|---|
Co-DINO | Swin-L | DETR | COCO | 64.1 | - | - | - | config | model |
Co-DINO | ViT-L | DETR | COCO | 65.9 | - | 66.0 | - | config | model |
Co-DINO | Swin-L | LSJ | LVIS | 64.5 | - | - | - | config (test) | model |
Co-DINO | ViT-L | LSJ | LVIS | 68.0 | - | - | - | config (test) | model |
Co-DINO-Inst | ViT-L | LSJ | LVIS | 67.3 | 60.7 | - | - | config (test) | model |
Model | Backbone | Epochs | Aug | Dataset | box AP | Config | Download |
---|---|---|---|---|---|---|---|
Co-DINO | R50 | 12 | DETR | COCO | 52.1 | config | model |
Co-DINO | R50 | 12 | LSJ | COCO | 52.1 | config | model |
Co-DINO-9enc | R50 | 12 | LSJ | COCO | 52.6 | config | model |
Co-DINO | R50 | 36 | LSJ | COCO | 54.8 | config | model |
Co-DINO-9enc | R50 | 36 | LSJ | COCO | 55.4 | config | model |
Model | Backbone | Epochs | Aug | Dataset | box AP | Config | Download |
---|---|---|---|---|---|---|---|
Co-DINO | Swin-L | 12 | DETR | COCO | 58.9 | config | model |
Co-DINO | Swin-L | 24 | DETR | COCO | 59.8 | config | model |
Co-DINO | Swin-L | 36 | DETR | COCO | 60.0 | config | model |
Co-DINO | Swin-L | 12 | LSJ | COCO | 59.3 | config | model |
Co-DINO | Swin-L | 24 | LSJ | COCO | 60.4 | config | model |
Co-DINO | Swin-L | 36 | LSJ | COCO | 60.7 | config | model |
Co-DINO | Swin-L | 36 | LSJ | LVIS | 56.9 | config (test) | model |
Model | Backbone | Epochs | Queries | box AP | Config | Download |
---|---|---|---|---|---|---|
Co-Deformable-DETR | R50 | 12 | 300 | 49.5 | config | model | log |
Co-Deformable-DETR | Swin-T | 12 | 300 | 51.7 | config | model | log |
Co-Deformable-DETR | Swin-T | 36 | 300 | 54.1 | config | model | log |
Co-Deformable-DETR | Swin-S | 12 | 300 | 53.4 | config | model | log |
Co-Deformable-DETR | Swin-S | 36 | 300 | 55.3 | config | model | log |
Co-Deformable-DETR | Swin-B | 12 | 300 | 55.5 | config | model | log |
Co-Deformable-DETR | Swin-B | 36 | 300 | 57.5 | config | model | log |
Co-Deformable-DETR | Swin-L | 12 | 300 | 56.9 | config | model | log |
Co-Deformable-DETR | Swin-L | 36 | 900 | 58.5 | config | model | log |
We implement Co-DETR using MMDetection V2.25.3 and MMCV V1.5.0.
The source code of MMdetection has been included in this repo and you only need to build MMCV following official instructions.
We test our models under python=3.7.11,pytorch=1.11.0,cuda=11.3
. Other versions may not be compatible.
The COCO dataset and LVIS dataset should be organized as:
Co-DETR
└── data
├── coco
│ ├── annotations
│ │ ├── instances_train2017.json
│ │ └── instances_val2017.json
│ ├── train2017
│ └── val2017
│
└── lvis_v1
├── annotations
│ ├── lvis_v1_train.json
│ └── lvis_v1_val.json
├── train2017
└── val2017
Train Co-Deformable-DETR + ResNet-50 with 8 GPUs:
sh tools/dist_train.sh projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py 8 path_to_exp
Train using slurm:
sh tools/slurm_train.sh partition job_name projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py path_to_exp
Test Co-Deformable-DETR + ResNet-50 with 8 GPUs, and evaluate:
sh tools/dist_test.sh projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py path_to_checkpoint 8 --eval bbox
Test using slurm:
sh tools/slurm_test.sh partition job_name projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py path_to_checkpoint --eval bbox
If you find this repository useful, please use the following BibTeX entry for citation.
@inproceedings{zong2023detrs,
title={Detrs with collaborative hybrid assignments training},
author={Zong, Zhuofan and Song, Guanglu and Liu, Yu},
booktitle={Proceedings of the IEEE/CVF international conference on computer vision},
pages={6748--6758},
year={2023}
}
This project is released under the MIT license. Please see the LICENSE file for more information.