Our paper The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generation has been accepted by CVPR 2022.
Check INSTALL.md for installation instructions.
Check DATASET.md for instructions of dataset preprocessing.
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --master_port 10025 --nproc_per_node=2 tools/relation_train_net.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX True MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True MODEL.ROI_RELATION_HEAD.PREDICTOR MotifConfidencePredictor SOLVER.IMS_PER_BATCH 12 TEST.IMS_PER_BATCH 2 DTYPE "float16" SOLVER.MAX_ITER 50000 SOLVER.VAL_PERIOD 2000 SOLVER.CHECKPOINT_PERIOD 2000 GLOVE_DIR glove MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/pretrained_faster_rcnn/model_final.pth OUTPUT_DIR checkpoints/motif-predcls-non-bg-reweight TYPE None ADD_BG False NEW_DATA_PATH new_data SOLVER.SCHEDULE.TYPE WarmupMultiStepLR SOLVER.BASE_LR 0.001
CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=1 python -m torch.distributed.launch --master_port 10037 --nproc_per_node=1 tools/generate_has_rels_results.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX True MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True MODEL.ROI_RELATION_HEAD.PREDICTOR MotifPredictor TEST.IMS_PER_BATCH 1 DTYPE "float16" GLOVE_DIR glove MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/motif-precls-exmp OUTPUT_DIR checkpoints/motif-precls-exmp GROUPS tail GROUPS_T [0.6] ADD_BG True NEW_DATA_PATH new_data
CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=1 python -m torch.distributed.launch --master_port 10031 --nproc_per_node=1 tools/generate_nsd_results.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX True MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True MODEL.ROI_RELATION_HEAD.PREDICTOR MotifConfidencePredictor TEST.IMS_PER_BATCH 1 DTYPE "float16" GLOVE_DIR glove MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/motif-predcls-non-bg-reweight OUTPUT_DIR checkpoints/motif-predcls-non-bg-reweight GROUPS tail GROUPS_T [0.6] TYPE complete_bg ADD_BG False NEW_DATA_PATH new_data
CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=1 python -m torch.distributed.launch --master_port 10032 --nproc_per_node=1 tools/generate_nice_results.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX True MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True MODEL.ROI_RELATION_HEAD.PREDICTOR MotifPredictor TEST.IMS_PER_BATCH 1 DTYPE "float16" GLOVE_DIR glove MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/motif-precls-exmp OUTPUT_DIR checkpoints/motif-precls-exmp GROUPS tail GROUPS_T [0.6] TYPE extract_pos ADD_BG True NEW_DATA_PATH new_data
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --master_port 10025 --nproc_per_node=2 tools/relation_train_net.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX True MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True MODEL.ROI_RELATION_HEAD.PREDICTOR MotifPredictor SOLVER.IMS_PER_BATCH 12 TEST.IMS_PER_BATCH 2 DTYPE "float16" SOLVER.MAX_ITER 50000 SOLVER.VAL_PERIOD 2000 SOLVER.CHECKPOINT_PERIOD 2000 GLOVE_DIR glove MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/pretrained_faster_rcnn/model_final.pth OUTPUT_DIR checkpoints/model_predcls_nice TYPE nice ADD_BG True NEW_DATA_PATH new_data
To make it easier for you to run our code, the Parameters in the command are explained here:
--master_port
: It represents the port on which the command is run.CUDA_VISIBLE_DEVICES
: It means the the GPUs that you are going to use. For example, CUDA_VISIBLE_DEVICES=0,1
use the first two GPUs.--nproc_per_node
: It is the number of GPUs you are going to use.SOLVER.IMS_PER_BATCH
: It is the training batch size.TEST.IMS_PER_BATCH
: It is the testing batch size.SOLVER.MAX_ITER
: It is the maximum iteration.SOLVER.STEPS
: It is the steps where we decay the learning rateSOLVER.VAL_PERIOD
: It is the period of conducting val.SOLVER.CHECKPOINT_PERIOD
: It is the period of saving checkpoint.MODEL.RELATION_ON
It means turning on the relationship head or not (since this is the pretraining phase for Faster R-CNN only, we turn off the relationship head), OUTPUT_DIR is the output directory to save checkpoints.MODEL.ROI_RELATION_HEAD.USE_GT_BOX
and MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL
: They used to select the protocols, (1) PredCls: They are all set as True
. (2) SGCls: MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL
is set to False
, while MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True
is set to True
. (3) SGDet: They are all set to False
.MODEL.ROI_RELATION_HEAD.PREDICTOR
: It is the backbobe you are going to use., and the MOTIFS SGG backbone (MotifPredictor
) is used by default. GROUPS
: It represents the predicate in which group to complete. The options are head
, body
, tail
, or their combinations, separated by commas. TYPE
: The type of dataset loaded. If it set to 'complete_bg', it represents the loading of triplet samples of 'background' predicates. If it set to 'extract_pos', it means the loading of the positive triplet samples. If it set to 'nice', it means the loading of the samples generated by NICE. GROUPS_T
: Thresholds of confidence score of groups in 'GROUPS'.ADD_BG
: It means whether or not the sample loaded contains 'background'.NEW_DATA_PATH
: The path where NICE generates new datasets.
For the Motifs Predictor in NSD and Motifs-NICE, we provide the trained models (checkpoint) for verification purpose. Please download from here* and unzip to checkpoints. Besides, we provide the files generated in new_data, you can download from here*.) The NSD checkpoint can be download from here*.
If you find this project helps your research, please kindly consider citing our project or papers in your publications.
Our codebase is based on Scene-Graph-Benchmark.pytorch.