About basetraining performance

[YAOSL98]

您好，我按照您给出的log对MSCOCO数据集进行了重新的base training，但是只用了一张Nvidia V100 GPU。重新训练后的base training model相较于您提供的MSCOCO的base training model性能下降很多，尤其是在base class。 Hello, I re-base trained the MSCOCO dataset according to the log you gave, but only used an Nvidia V100 GPU. Compared with the base training model of MSCOCO provided by you, the performance of the base training model after retraining decreases a lot, especially in the base class.

我使用的base training的参数 I used these parameters of base training

EXP_DIR=exps/coco BASE_TRAIN_DIR=${EXP_DIR}/base_train mkdir exps mkdir ${EXP_DIR} mkdir ${BASE_TRAIN_DIR}

python -u main.py \ --dataset_file coco_base \ --backbone resnet101 \ --num_feature_levels 1 \ --enc_layers 6 \ --dec_layers 6 \ --hidden_dim 256 \ --num_queries 300 \ --batch_size 4 \ --category_codes_cls_loss \ --epoch 25 \ --lr_drop_milestones 20 \ --save_every_epoch 5 \ --eval_every_epoch 5 \ --output_dir ${BASE_TRAIN_DIR} \ 2>&1 | tee ${BASE_TRAIN_DIR}/log.txt

重新train后的base training model： base training model after re-train:

• Novel Categories: Accumulating evaluation results... DONE (t=3.51s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.002 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.006 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.001 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.011 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.044 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.066 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.052 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.120

  • Base Categories: Accumulating evaluation results... DONE (t=9.94s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.002 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.008 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.001 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.006 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.014 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.050 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.072 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.006 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.061 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.157

主页上提供的base training model： base training model provided on this github:

• Base Categories: Accumulating evaluation results... DONE (t=10.00s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.346 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.538 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.365 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.183 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.403 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.516 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.314 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.496 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.530 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.303 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.603 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.764
• Novel Categories: Accumulating evaluation results... DONE (t=6.01s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.006 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.012 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.005 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.008 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.010 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.050 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.105 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.110 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.067 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.102 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.197

[ZhangGongjie]

It's not surprising. If you train the original Deformable DETR with only batch_size = 2, the performance will drop compared to their paper.

[Caltech-Z]

Q1：ImportError: cannot import name '_NewEmptyTensorOp' from 'torchvision.ops.misc' Q2：PermissionError: [Errno 13] Permission denied: './basetrain.sh' For the above two questions, can the owner answer and solve them?