conda create -n BLC python=3.7 -y
conda activate BLC
conda install pytorch=1.1.0 torchvision=0.3.0 cudatoolkit=10.0 -c pytorch
pip install cython && pip --no-cache-dir install -r requirements.txt
python setup.py develop
Download the train and test annotations files for BLC from annotations, put all json label file to
data/coco/annotations/
Download MSCOCO-2014 dataset and unzip the images it to path:
data/coco/train2014/
data/coco/val2014/
training:
train BLRPN:
48/17 split:
./tools/dist_train.sh configs/train/BLRPN/blrpn_r50_fpn_semantic_48_17_1x.py 4
65/15 split:
./tools/dist_train.sh configs/train/BLRPN/blrpn_r50_fpn_semantic_65_15_1x.py 4
generate new Ws:
48/17 split:
python tools/replace_bg_w2vec_48_17.py
65/15 split:
python tools/replace_bg_w2vec_65_15.py
train Cascade Semantic R-CNN with new Ws:
./tools/dist_train.sh configs/BLC/train/Cascade_Semantic_R-CNN/cascade_semantic_rcnn_information_flow_learnable_bg_48_17_1x.py 4
./tools/dist_train.sh configs/BLC/train/Cascade_Semantic_R-CNN/cascade_semantic_rcnn_information_flow_learnable_bg_48_17_1x.py 4
Inference & Evaluate:
ZSD task:
48/17 split ZSD task:
download ms 48/17 BLC ms model, put it in checkpoints/BLC_ms_48_17.pth
inference:
python tools/test.py configs/BLC/inference/zsd/cascade_semantic_rcnn_information_flow_learnable_bg_48_17_1x.py checkpoints/BLC_ms_48_17.pth --out results/zsd_48_17.pkl
our results zsd_48_17.pkl can also downloaded from zsd_48_17.pkl.
evaluate:
python tools/zsd_eval.py results/zsd_48_17.pkl configs/BLC/inference/zsd/cascade_semantic_rcnn_information_flow_learnable_bg_48_17_1x.py
65/15 split ZSD task:
download ms 65/15 BLC model, put it in checkpoints/BLC_ms_65_15.pth
inference:
python tools/test.py configs/BLC/inference/zsd/cascade_semantic_rcnn_information_flow_learnable_bg_65_15_1x.py checkpoints/BLC_ms_65_15.pth --out results/zsd_65_15.pkl
our results zsd_65_15.pkl can also downloaded from zsd_65_15.pkl.
evaluate:
python tools/zsd_eval.py results/zsd_65_15.pkl configs/BLC/inference/zsd/cascade_semantic_rcnn_information_flow_learnable_bg_65_15_1x.py
GZSD task:
48/17 split GZSD task:
download 48/17 BLC model, put it in checkpoints/BLC_48_17.pth
inference:
python tools/test.py configs/BLC/inference/gzsd/gzsd_cascade_semantic_rcnn_information_flow_learnable_bg_48_17_1x.py checkpoints/BLC_48_17.pth --out results/gzsd_48_17.pkl
our results gzsd_48_17.pkl can also downloaded from gzsd_48_17.pkl.
evaluate:
python tools/gzsd_eval.py results/gzsd_48_17.pkl configs/BLC/inference/gzsd/gzsd_cascade_semantic_rcnn_information_flow_learnable_bg_48_17_1x.py
65/15 split ZSD task:
python tools/test.py configs/BLC/inference/gzsd/gzsd_cascade_semantic_rcnn_information_flow_learnable_bg_65_15_1x.py checkpoints/BLC_65_15.pth --out results/gzsd_65_15.pkl
python tools/gzsd_eval.py results/gzsd_65_15.pkl configs/BLC/inference/gzsd/gzsd_cascade_semantic_rcnn_information_flow_learnable_bg_65_15_1x.py
ZSD is released under MIT License.
If you use BLC in your research or wish to refer to the baseline results published here, please use the following BibTeX entries:
@InProceedings{Zheng_2020_ACCV,
author = {Zheng, Ye and Huang, Ruoran and Han, Chuanqi and Huang, Xi and Cui, Li},
title = {Background Learnable Cascade for Zero-Shot Object Detection},
booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)},
month = {November},
year = {2020}
}