AP low

[xiaotuoluo11]

train directives： python -m torch.distributed.launch --nproc_per_node=2 tools/train_net.py --config-file configs/pretrain/glip_Swin_T_O365_GoldG.yaml --skip-test MODEL.WEIGHT tools/111/在次上传/glip_a_tiny _o365.pth DATASETS.TRAIN '("coco_grounding_train", )' MODEL.BACKBONE.FREEZE_CONV_BODY_AT -1 SOLVER.IMS_PER_BATCH 8 SOLVER.USE_AMP True SOLVER.MAX_EPOCH 10 TEST.DURING_TRAINING False TEST. IMS_PER_BATCH 6 SOLVER.FIND_UNUSED_PARAMETERS False SOLVER.BASE_LR 0.001 SOLVER.LANG_LR 0.001 SOLVER.STEPS (0.67,0.89) DATASETS.DISABLE_SHUFFLE True MODEL.DYHEAD.SCORE_AGG "MEAN" TEST.E VAL_TASK detection

train log : 3958 eta: 0:06:59 iter: 43800 loss: 1.7392 (1.7894) loss_centerness: 0.6013 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8164 (0.8410) loss_reg: 0.2709 (0.3397) tim e: 2.3483 (2.4086) data: 0.0217 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3959 eta: 0:06:10 iter: 43820 loss: 1.7498 (1.7894) loss_centerness: 0.6035 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8415 (0.8410) loss_reg: 0.2941 (0.3397) tim e: 2.4378 (2.4086) data: 0.0213 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3960 eta: 0:05:22 iter: 43840 loss: 1.7865 (1.7894) loss_centerness: 0.6088 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8234 (0.8410) loss_reg: 0.3513 (0.3397) tim e: 2.4683 (2.4087) data: 0.0216 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3961 eta: 0:04:34 iter: 43860 loss: 1.7520 (1.7894) loss_centerness: 0.6046 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8272 (0.8410) loss_reg: 0.3083 (0.3397) tim e: 2.3602 (2.4086) data: 0.0216 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3962 eta: 0:03:46 iter: 43880 loss: 1.7564 (1.7893) loss_centerness: 0.6054 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8171 (0.8410) loss_reg: 0.3195 (0.3397) tim e: 2.4565 (2.4087) data: 0.0213 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3963 eta: 0:02:58 iter: 43900 loss: 1.7476 (1.7893) loss_centerness: 0.6073 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8139 (0.8410) loss_reg: 0.3038 (0.3396) tim e: 2.3082 (2.4086) data: 0.0211 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3964 eta: 0:02:10 iter: 43920 loss: 1.7471 (1.7893) loss_centerness: 0.6095 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8209 (0.8410) loss_reg: 0.3259 (0.3396) tim e: 2.4580 (2.4086) data: 0.0217 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3965 eta: 0:01:21 iter: 43940 loss: 1.7128 (1.7893) loss_centerness: 0.6020 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8070 (0.8410) loss_reg: 0.2810 (0.3396) tim e: 2.3065 (2.4086) data: 0.0209 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3966 eta: 0:00:33 iter: 43960 loss: 1.7846 (1.7893) loss_centerness: 0.6009 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8406 (0.8410) loss_reg: 0.3147 (0.3396) tim e: 2.4452 (2.4086) data: 0.0213 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209 3967 eta: 0:00:00 iter: 43974 loss: 1.7516 (1.7893) loss_centerness: 0.6062 (0.6087) loss_cls: 0.0000 (0.0000) loss_dot_product_token: 0.8295 (0.8410) loss_reg: 0.3060 (0.3396) tim e: 2.3669 (2.4086) data: 0.0215 (0.0227) lr: 0.000001 wd: 0.000100 max mem: 28209

eval directives: python tools/test_grounding_net.py --config-file configs/pretrain/glip_Swin_T_O365_GoldG.yaml --weight OUTPUT/model_0042500.pth TEST.IMS_PER_BATCH 1 MODEL.DYHEAD.SCORE_AGG "MEAN" TEST.EVAL_TASK detection MODEL.DYHEAD.FUSE_CONFIG.MLM_LOSS False OUTPUT_DIR coco_result eval result : 2023-06-25 13:06:27,010 maskrcnn_benchmark.inference INFO: OrderedDict([('bbox', OrderedDict([('AP', 0.01649729253736016), ('AP50', 0.01118038977604), ('AP75', 0.0179191313029086), ( 'APs', 0.03604823204377465), ('APm', 0.0363606871560514), ('APl', 0.0164767005869501)]))])

### Tasks
- [ ] question：Why is AP so low?

[liunian-harold-li]

Hi there, if you want to use GLIP_T_A checkpoints, the config should also be changed to glip_a configs (https://github.com/microsoft/GLIP/blob/main/configs/pretrain/glip_A_Swin_T_O365.yaml). The glip_Swin_T_O365_GoldG.yaml config has a different model structure.

[zilong69]

Why is the loss_cls 0?I also had the same problem

[lixi92]

i find the code below: if self.cfg.MODEL.DYHEAD.FUSE_CONFIG.USE_CLASSIFICATION_LOSS: losses["loss_cls"] = loss_box_cls else: losses["loss_cls"] = 0.0 * loss_box_cls

then i find MODEL.DYHEAD.FUSE_CONFIG.USE_CLASSIFICATION_LOSS is set to be false in the default config file and checkpoint config file why?

[liunian-harold-li]

Sorry for the delay in response! The USE_CLASSIFICATION_LOSS means using the original fixed-class classification loss; it is replaced by loss_dot_product_token, as now we are doing a classification over the input language tokens.