search

ChenhongyiYang / QueryDet-PyTorch

[CVPR 2022 Oral] QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection
MIT License
432 stars 47 forks source link

运行报错 #17

Closed XuKer closed 2 years ago

XuKer commented 2 years ago

image 请问下，我运行的时候一直报这个错是什么原因

ChenhongyiYang commented 2 years ago

Hi. I think this error is caused by the updated APEX. Can you please comment out this line and try again?

XuKer commented 2 years ago

Hi. I think this error is caused by the updated APEX. Can you please comment out this line and try again? image I commented out this line and it can run successfully, but the test output has no value here

ChenhongyiYang commented 2 years ago

Hi, can you please tell me what command you ran?

XuKer commented 2 years ago

Hi, can you please tell me what command you ran?

python train_coco.py --config-file models/querydet/configs/coco/train.yaml --num-gpu 1 OUTPUT_DIR /output

XuKer commented 2 years ago

image

XuKer commented 2 years ago

Hi, can you please tell me what command you ran?

retinant 可以正常train

ChenhongyiYang commented 2 years ago

Hi. This problem is weird. Can you please copy-paste the last part of the training log?

XuKer commented 2 years ago
[34manchor_generator.cell_anchors.{0, 1, 2, 3, 4, 5} backbone.bottom_up.res2.0.conv1.norm.{running_mean, running_var} backbone.bottom_up.res2.0.conv2.norm.{running_mean, running_var} backbone.bottom_up.res2.0.conv3.norm.{running_mean, running_var} backbone.bottom_up.res2.0.shortcut.norm.{running_mean, running_var} backbone.bottom_up.res2.1.conv1.norm.{running_mean, running_var} backbone.bottom_up.res2.1.conv2.norm.{running_mean, running_var} backbone.bottom_up.res2.1.conv3.norm.{running_mean, running_var} backbone.bottom_up.res2.2.conv1.norm.{running_mean, running_var} backbone.bottom_up.res2.2.conv2.norm.{running_mean, running_var} backbone.bottom_up.res2.2.conv3.norm.{running_mean, running_var} backbone.bottom_up.res3.0.conv1.norm.{running_mean, running_var} backbone.bottom_up.res3.0.conv2.norm.{running_mean, running_var} backbone.bottom_up.res3.0.conv3.norm.{running_mean, running_var} backbone.bottom_up.res3.0.shortcut.norm.{running_mean, running_var} backbone.bottom_up.res3.1.conv1.norm.{running_mean, running_var} backbone.bottom_up.res3.1.conv2.norm.{running_mean, running_var} backbone.bottom_up.res3.1.conv3.norm.{running_mean, running_var} backbone.bottom_up.res3.2.conv1.norm.{running_mean, running_var} backbone.bottom_up.res3.2.conv2.norm.{running_mean, running_var} backbone.bottom_up.res3.2.conv3.norm.{running_mean, running_var} backbone.bottom_up.res3.3.conv1.norm.{running_mean, running_var} backbone.bottom_up.res3.3.conv2.norm.{running_mean, running_var} backbone.bottom_up.res3.3.conv3.norm.{running_mean, running_var} backbone.bottom_up.res4.0.conv1.norm.{running_mean, running_var} backbone.bottom_up.res4.0.conv2.norm.{running_mean, running_var} backbone.bottom_up.res4.0.conv3.norm.{running_mean, running_var} backbone.bottom_up.res4.0.shortcut.norm.{running_mean, running_var} backbone.bottom_up.res4.1.conv1.norm.{running_mean, running_var} backbone.bottom_up.res4.1.conv2.norm.{running_mean, running_var} backbone.bottom_up.res4.1.conv3.norm.{running_mean, running_var} backbone.bottom_up.res4.2.conv1.norm.{running_mean, running_var} backbone.bottom_up.res4.2.conv2.norm.{running_mean, running_var} backbone.bottom_up.res4.2.conv3.norm.{running_mean, running_var} backbone.bottom_up.res4.3.conv1.norm.{running_mean, running_var} backbone.bottom_up.res4.3.conv2.norm.{running_mean, running_var} backbone.bottom_up.res4.3.conv3.norm.{running_mean, running_var} backbone.bottom_up.res4.4.conv1.norm.{running_mean, running_var} backbone.bottom_up.res4.4.conv2.norm.{running_mean, running_var} backbone.bottom_up.res4.4.conv3.norm.{running_mean, running_var} backbone.bottom_up.res4.5.conv1.norm.{running_mean, running_var} backbone.bottom_up.res4.5.conv2.norm.{running_mean, running_var} backbone.bottom_up.res4.5.conv3.norm.{running_mean, running_var} backbone.bottom_up.res5.0.conv1.norm.{running_mean, running_var} backbone.bottom_up.res5.0.conv2.norm.{running_mean, running_var} backbone.bottom_up.res5.0.conv3.norm.{running_mean, running_var} backbone.bottom_up.res5.0.shortcut.norm.{running_mean, running_var} backbone.bottom_up.res5.1.conv1.norm.{running_mean, running_var} backbone.bottom_up.res5.1.conv2.norm.{running_mean, running_var} backbone.bottom_up.res5.1.conv3.norm.{running_mean, running_var} backbone.bottom_up.res5.2.conv1.norm.{running_mean, running_var} backbone.bottom_up.res5.2.conv2.norm.{running_mean, running_var} backbone.bottom_up.res5.2.conv3.norm.{running_mean, running_var} backbone.bottom_up.stem.conv1.norm.{running_mean, running_var} backbone.fpn_lateral2.{bias, weight} backbone.fpn_lateral3.{bias, weight} backbone.fpn_lateral4.{bias, weight} backbone.fpn_lateral5.{bias, weight} backbone.fpn_output2.{bias, weight} backbone.fpn_output3.{bias, weight} backbone.fpn_output4.{bias, weight} backbone.fpn_output5.{bias, weight} backbone.top_block.p6.{bias, weight} backbone.top_block.p7.{bias, weight} det_head.bbox_layer_0.{bias, weight} det_head.bbox_layer_1.{bias, weight} det_head.bbox_layer_2.{bias, weight} det_head.bbox_layer_3.{bias, weight} det_head.bbox_pred.{bias, weight} det_head.cls_layer_0.{bias, weight} det_head.cls_layer_1.{bias, weight} det_head.cls_layer_2.{bias, weight} det_head.cls_layer_3.{bias, weight} det_head.cls_score.{bias, weight} pixel_mean pixel_std query_anchor_generator.cell_anchors.{0, 1} query_head.layer_0.{bias, weight} query_head.layer_1.{bias, weight} query_head.layer_2.{bias, weight} query_head.layer_3.{bias, weight} query_head.pred_net.{bias, weight} [04/14 09:02:45] d2.checkpoint.c2_model_loading INFO: The checkpoint state_dict contains keys that are not used by the model: fc1000_b fc1000_w res2_0_branch2a_b res2_0_branch2b_b res2_0_branch2c_b res2_0_branch1_b res2_1_branch2a_b res2_1_branch2b_b res2_1_branch2c_b res2_2_branch2a_b res2_2_branch2b_b res2_2_branch2c_b res3_0_branch2a_b res3_0_branch2b_b res3_0_branch2c_b res3_0_branch1_b res3_1_branch2a_b res3_1_branch2b_b res3_1_branch2c_b res3_2_branch2a_b res3_2_branch2b_b res3_2_branch2c_b res3_3_branch2a_b res3_3_branch2b_b res3_3_branch2c_b res4_0_branch2a_b res4_0_branch2b_b res4_0_branch2c_b res4_0_branch1_b res4_1_branch2a_b res4_1_branch2b_b res4_1_branch2c_b res4_2_branch2a_b res4_2_branch2b_b res4_2_branch2c_b res4_3_branch2a_b res4_3_branch2b_b res4_3_branch2c_b res4_4_branch2a_b res4_4_branch2b_b res4_4_branch2c_b res4_5_branch2a_b res4_5_branch2b_b res4_5_branch2c_b res5_0_branch2a_b res5_0_branch2b_b res5_0_branch2c_b res5_0_branch1_b res5_1_branch2a_b res5_1_branch2b_b res5_1_branch2c_b res5_2_branch2a_b res5_2_branch2b_b res5_2_branch2c_b conv1_b [04/14 09:03:01] d2.data.datasets.coco INFO: Loading /data/coco/annotations/instances_train2017.json takes 15.27 seconds. [04/14 09:03:02] d2.data.datasets.coco INFO: Loaded 118287 images in COCO format from /data/coco/annotations/instances_train2017.json [04/14 09:03:10] d2.data.build INFO: Removed 1021 images with no usable annotations. 117266 images left. [04/14 09:03:14] d2.data.build INFO: Distribution of instances among all 80 categories:  category #instances category #instances category #instances
person 257253 bicycle 7056 car 43533
motorcycle 8654 airplane 5129 bus 6061
train 4570 truck 9970 boat 10576
traffic light 12842 fire hydrant 1865 stop sign 1983
parking meter 1283 bench 9820 bird 10542
cat 4766 dog 5500 horse 6567
sheep 9223 cow 8014 elephant 5484
bear 1294 zebra 5269 giraffe 5128
backpack 8714 umbrella 11265 handbag 12342
tie 6448 suitcase 6112 frisbee 2681
skis 6623 snowboard 2681 sports ball 6299
kite 8802 baseball bat 3273 baseball gl.. 3747
skateboard 5536 surfboard 6095 tennis racket 4807
bottle 24070 wine glass 7839 cup 20574
fork 5474 knife 7760 spoon 6159
bowl 14323 banana 9195 apple 5776
sandwich 4356 orange 6302 broccoli 7261
carrot 7758 hot dog 2884 pizza 5807
donut 7005 cake 6296 chair 38073
couch 5779 potted plant 8631 bed 4192
dining table 15695 toilet 4149 tv 5803
laptop 4960 mouse 2261 remote 5700
keyboard 2854 cell phone 6422 microwave 1672
oven 3334 toaster 225 sink 5609
refrigerator 2634 book 24077 clock 6320
vase 6577 scissors 1464 teddy bear 4729
hair drier 198 toothbrush 1945
total 849949 
[04/14 09:03:14] d2.data.common INFO: Serializing 117266 elements to byte tensors and concatenating them all ... [04/14 09:03:17] d2.data.common INFO: Serialized dataset takes 453.67 MiB [04/14 09:03:17] d2.data.dataset_mapper INFO: Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()] [04/14 09:03:18] d2.data.build INFO: Using training sampler TrainingSampler [04/14 09:03:22] detectron2 INFO: Using APEX mixed precision trianing, opt_level=O1 [04/14 09:03:40] d2.utils.events INFO: eta: 0:13:15 iter: 19 total_loss: 10.403 loss_cls: 3.375 loss_box_reg: 1.376 loss_query: 5.542 time: 0.8045 data_time: 0.0492 lr: 0.000200 max_mem: 27843M [04/14 09:03:58] d2.utils.events INFO: eta: 0:13:10 iter: 39 total_loss: 9.524 loss_cls: 2.782 loss_box_reg: 1.082 loss_query: 5.591 time: 0.8406 data_time: 0.0251 lr: 0.000400 max_mem: 28515M [04/14 09:04:13] d2.utils.events INFO: eta: 0:12:36 iter: 59 total_loss: 9.153 loss_cls: 2.501 loss_box_reg: 0.986 loss_query: 5.508 time: 0.8188 data_time: 0.0173 lr: 0.000599 max_mem: 28515M [04/14 09:04:29] d2.utils.events INFO: eta: 0:12:20 iter: 79 total_loss: 9.080 loss_cls: 2.437 loss_box_reg: 0.979 loss_query: 5.561 time: 0.8175 data_time: 0.0215 lr: 0.000799 max_mem: 28515M [04/14 09:04:46] d2.utils.events INFO: eta: 0:12:08 iter: 99 total_loss: 9.106 loss_cls: 2.484 loss_box_reg: 0.977 loss_query: 5.543 time: 0.8159 data_time: 0.0197 lr: 0.000999 max_mem: 28515M [04/14 09:05:02] d2.utils.events INFO: eta: 0:11:54 iter: 119 total_loss: 9.214 loss_cls: 2.549 loss_box_reg: 1.002 loss_query: 5.513 time: 0.8153 data_time: 0.0220 lr: 0.001199 max_mem: 28515M [04/14 09:05:18] d2.utils.events INFO: eta: 0:11:36 iter: 139 total_loss: 8.534 loss_cls: 2.141 loss_box_reg: 0.876 loss_query: 5.547 time: 0.8106 data_time: 0.0209 lr: 0.001399 max_mem: 28515M [04/14 09:05:34] d2.utils.events INFO: eta: 0:11:23 iter: 159 total_loss: 9.459 loss_cls: 2.819 loss_box_reg: 1.144 loss_query: 5.556 time: 0.8132 data_time: 0.0272 lr: 0.001598 max_mem: 28515M [04/14 09:05:50] d2.utils.events INFO: eta: 0:11:08 iter: 179 total_loss: 9.233 loss_cls: 2.562 loss_box_reg: 1.061 loss_query: 5.556 time: 0.8125 data_time: 0.0207 lr: 0.001798 max_mem: 28515M [04/14 09:06:06] d2.utils.events INFO: eta: 0:10:52 iter: 199 total_loss: 8.972 loss_cls: 2.394 loss_box_reg: 0.982 loss_query: 5.538 time: 0.8120 data_time: 0.0218 lr: 0.001998 max_mem: 28515M [04/14 09:06:23] d2.utils.events INFO: eta: 0:10:36 iter: 219 total_loss: 8.772 loss_cls: 2.274 loss_box_reg: 0.892 loss_query: 5.507 time: 0.8113 data_time: 0.0220 lr: 0.002198 max_mem: 28515M [04/14 09:06:39] d2.utils.events INFO: eta: 0:10:22 iter: 239 total_loss: 9.330 loss_cls: 2.590 loss_box_reg: 1.011 loss_query: 5.710 time: 0.8112 data_time: 0.0255 lr: 0.002398 max_mem: 28515M [04/14 09:06:55] d2.utils.events INFO: eta: 0:10:06 iter: 259 total_loss: 8.719 loss_cls: 2.254 loss_box_reg: 0.857 loss_query: 5.564 time: 0.8112 data_time: 0.0251 lr: 0.002597 max_mem: 28515M [04/14 09:07:11] d2.utils.events INFO: eta: 0:09:50 iter: 279 total_loss: 9.144 loss_cls: 2.510 loss_box_reg: 0.983 loss_query: 5.565 time: 0.8114 data_time: 0.0210 lr: 0.002797 max_mem: 28515M [04/14 09:07:28] d2.utils.events INFO: eta: 0:09:34 iter: 299 total_loss: 8.830 loss_cls: 2.325 loss_box_reg: 0.927 loss_query: 5.614 time: 0.8128 data_time: 0.0211 lr: 0.002997 max_mem: 28515M [04/14 09:07:44] d2.utils.events INFO: eta: 0:09:18 iter: 319 total_loss: 9.197 loss_cls: 2.563 loss_box_reg: 1.050 loss_query: 5.548 time: 0.8135 data_time: 0.0246 lr: 0.003197 max_mem: 28515M [04/14 09:08:00] d2.utils.events INFO: eta: 0:09:02 iter: 339 total_loss: 9.310 loss_cls: 2.653 loss_box_reg: 1.039 loss_query: 5.543 time: 0.8127 data_time: 0.0241 lr: 0.003397 max_mem: 28515M [04/14 09:08:17] d2.utils.events INFO: eta: 0:08:46 iter: 359 total_loss: 9.337 loss_cls: 2.780 loss_box_reg: 1.100 loss_query: 5.527 time: 0.8143 data_time: 0.0240 lr: 0.003596 max_mem: 28515M [04/14 09:08:34] d2.utils.events INFO: eta: 0:08:30 iter: 379 total_loss: 9.079 loss_cls: 2.497 loss_box_reg: 1.022 loss_query: 5.624 time: 0.8142 data_time: 0.0222 lr: 0.003796 max_mem: 28515M [04/14 09:08:50] d2.utils.events INFO: eta: 0:08:14 iter: 399 total_loss: 8.823 loss_cls: 2.411 loss_box_reg: 0.953 loss_query: 5.551 time: 0.8151 data_time: 0.0221 lr: 0.003996 max_mem: 28515M [04/14 09:09:06] d2.utils.events INFO: eta: 0:07:57 iter: 419 total_loss: 9.163 loss_cls: 2.583 loss_box_reg: 1.016 loss_query: 5.565 time: 0.8140 data_time: 0.0167 lr: 0.004196 max_mem: 28515M [04/14 09:09:23] d2.utils.events INFO: eta: 0:07:41 iter: 439 total_loss: 9.232 loss_cls: 2.490 loss_box_reg: 0.973 loss_query: 5.553 time: 0.8155 data_time: 0.0212 lr: 0.004396 max_mem: 28515M [04/14 09:09:39] d2.utils.events INFO: eta: 0:07:24 iter: 459 total_loss: 9.278 loss_cls: 2.602 loss_box_reg: 1.007 loss_query: 5.497 time: 0.8153 data_time: 0.0170 lr: 0.004595 max_mem: 28515M [04/14 09:09:56] d2.utils.events INFO: eta: 0:07:08 iter: 479 total_loss: 9.499 loss_cls: 2.799 loss_box_reg: 1.139 loss_query: 5.567 time: 0.8156 data_time: 0.0187 lr: 0.004795 max_mem: 28515M [04/14 09:10:12] d2.utils.events INFO: eta: 0:06:51 iter: 499 total_loss: 8.783 loss_cls: 2.318 loss_box_reg: 0.910 loss_query: 5.534 time: 0.8153 data_time: 0.0213 lr: 0.004995 max_mem: 28515M [04/14 09:10:28] d2.utils.events INFO: eta: 0:06:34 iter: 519 total_loss: 9.293 loss_cls: 2.626 loss_box_reg: 1.050 loss_query: 5.496 time: 0.8141 data_time: 0.0167 lr: 0.005195 max_mem: 28515M [04/14 09:10:43] d2.utils.events INFO: eta: 0:06:18 iter: 539 total_loss: 9.220 loss_cls: 2.486 loss_box_reg: 1.124 loss_query: 5.552 time: 0.8130 data_time: 0.0230 lr: 0.005395 max_mem: 28515M [04/14 09:10:59] d2.utils.events INFO: eta: 0:06:01 iter: 559 total_loss: 9.347 loss_cls: 2.674 loss_box_reg: 1.064 loss_query: 5.566 time: 0.8127 data_time: 0.0189 lr: 0.005594 max_mem: 28515M [04/14 09:11:16] d2.utils.events INFO: eta: 0:05:45 iter: 579 total_loss: 9.074 loss_cls: 2.477 loss_box_reg: 0.967 loss_query: 5.570 time: 0.8130 data_time: 0.0246 lr: 0.005794 max_mem: 28515M [04/14 09:11:32] d2.utils.events INFO: eta: 0:05:28 iter: 599 total_loss: 9.332 loss_cls: 2.665 loss_box_reg: 1.071 loss_query: 5.547 time: 0.8123 data_time: 0.0206 lr: 0.005994 max_mem: 28515M [04/14 09:11:48] d2.utils.events INFO: eta: 0:05:12 iter: 619 total_loss: 8.971 loss_cls: 2.540 loss_box_reg: 1.013 loss_query: 5.559 time: 0.8120 data_time: 0.0192 lr: 0.006194 max_mem: 28515M [04/14 09:12:04] d2.utils.events INFO: eta: 0:04:56 iter: 639 total_loss: 8.915 loss_cls: 2.410 loss_box_reg: 0.948 loss_query: 5.518 time: 0.8128 data_time: 0.0202 lr: 0.006394 max_mem: 28515M [04/14 09:12:21] d2.utils.events INFO: eta: 0:04:39 iter: 659 total_loss: 8.753 loss_cls: 2.285 loss_box_reg: 0.895 loss_query: 5.588 time: 0.8127 data_time: 0.0193 lr: 0.006593 max_mem: 28515M [04/14 09:12:37] d2.utils.events INFO: eta: 0:04:23 iter: 679 total_loss: 8.921 loss_cls: 2.439 loss_box_reg: 0.991 loss_query: 5.521 time: 0.8131 data_time: 0.0250 lr: 0.006793 max_mem: 28515M [04/14 09:12:53] d2.utils.events INFO: eta: 0:04:06 iter: 699 total_loss: 9.235 loss_cls: 2.709 loss_box_reg: 1.082 loss_query: 5.594 time: 0.8131 data_time: 0.0173 lr: 0.006993 max_mem: 28515M [04/14 09:13:10] d2.utils.events INFO: eta: 0:03:50 iter: 719 total_loss: 8.992 loss_cls: 2.439 loss_box_reg: 0.974 loss_query: 5.532 time: 0.8131 data_time: 0.0167 lr: 0.007193 max_mem: 28515M [04/14 09:13:26] d2.utils.events INFO: eta: 0:03:33 iter: 739 total_loss: 8.830 loss_cls: 2.422 loss_box_reg: 0.940 loss_query: 5.469 time: 0.8132 data_time: 0.0201 lr: 0.007393 max_mem: 28515M [04/14 09:13:42] d2.utils.events INFO: eta: 0:03:17 iter: 759 total_loss: 9.219 loss_cls: 2.592 loss_box_reg: 1.000 loss_query: 5.637 time: 0.8133 data_time: 0.0212 lr: 0.007592 max_mem: 28515M [04/14 09:13:59] d2.utils.events INFO: eta: 0:03:01 iter: 779 total_loss: 9.529 loss_cls: 2.810 loss_box_reg: 1.156 loss_query: 5.543 time: 0.8134 data_time: 0.0184 lr: 0.007792 max_mem: 28515M [04/14 09:14:15] d2.utils.events INFO: eta: 0:02:44 iter: 799 total_loss: 9.176 loss_cls: 2.619 loss_box_reg: 1.125 loss_query: 5.491 time: 0.8135 data_time: 0.0200 lr: 0.007992 max_mem: 28515M [04/14 09:14:31] d2.utils.events INFO: eta: 0:02:28 iter: 819 total_loss: 8.908 loss_cls: 2.490 loss_box_reg: 0.986 loss_query: 5.535 time: 0.8130 data_time: 0.0220 lr: 0.008192 max_mem: 28515M [04/14 09:14:47] d2.utils.events INFO: eta: 0:02:12 iter: 839 total_loss: 9.297 loss_cls: 2.677 loss_box_reg: 1.125 loss_query: 5.510 time: 0.8127 data_time: 0.0181 lr: 0.008392 max_mem: 28515M [04/14 09:15:03] d2.utils.events INFO: eta: 0:01:55 iter: 859 total_loss: 9.083 loss_cls: 2.531 loss_box_reg: 0.988 loss_query: 5.550 time: 0.8124 data_time: 0.0168 lr: 0.008591 max_mem: 28515M [04/14 09:15:20] d2.utils.events INFO: eta: 0:01:39 iter: 879 total_loss: 9.080 loss_cls: 2.613 loss_box_reg: 1.020 loss_query: 5.546 time: 0.8129 data_time: 0.0193 lr: 0.008791 max_mem: 28515M [04/14 09:15:36] d2.utils.events INFO: eta: 0:01:22 iter: 899 total_loss: 9.088 loss_cls: 2.512 loss_box_reg: 1.003 loss_query: 5.585 time: 0.8124 data_time: 0.0221 lr: 0.008991 max_mem: 28515M [04/14 09:15:52] d2.utils.events INFO: eta: 0:01:06 iter: 919 total_loss: 9.152 loss_cls: 2.519 loss_box_reg: 1.030 loss_query: 5.533 time: 0.8123 data_time: 0.0203 lr: 0.009191 max_mem: 28515M [04/14 09:16:08] d2.utils.events INFO: eta: 0:00:50 iter: 939 total_loss: 9.343 loss_cls: 2.596 loss_box_reg: 1.056 loss_query: 5.581 time: 0.8127 data_time: 0.0187 lr: 0.009391 max_mem: 28515M [04/14 09:16:24] d2.utils.events INFO: eta: 0:00:33 iter: 959 total_loss: 8.697 loss_cls: 2.112 loss_box_reg: 0.850 loss_query: 5.600 time: 0.8124 data_time: 0.0165 lr: 0.009590 max_mem: 28515M [04/14 09:16:41] d2.utils.events INFO: eta: 0:00:17 iter: 979 total_loss: 9.286 loss_cls: 2.662 loss_box_reg: 1.046 loss_query: 5.579 time: 0.8125 data_time: 0.0182 lr: 0.009790 max_mem: 28515M [04/14 09:16:57] fvcore.common.checkpoint INFO: Saving checkpoint to /qdnet/outqd2/model_final.pth [04/14 09:16:58] d2.data.datasets.coco INFO: Loaded 5000 images in COCO format from /data/coco/annotations/instances_val2017.json [04/14 09:16:58] d2.data.build INFO: Distribution of instances among all 80 categories:  category #instances category #instances category #instances
person 10777 bicycle 314 car 1918
motorcycle 367 airplane 143 bus 283
train 190 truck 414 boat 424
traffic light 634 fire hydrant 101 stop sign 75
parking meter 60 bench 411 bird 427
cat 202 dog 218 horse 272
sheep 354 cow 372 elephant 252
bear 71 zebra 266 giraffe 232
backpack 371 umbrella 407 handbag 540
tie 252 suitcase 299 frisbee 115
skis 241 snowboard 69 sports ball 260
kite 327 baseball bat 145 baseball gl.. 148
skateboard 179 surfboard 267 tennis racket 225
bottle 1013 wine glass 341 cup 895
fork 215 knife 325 spoon 253
bowl 623 banana 370 apple 236
sandwich 177 orange 285 broccoli 312
carrot 365 hot dog 125 pizza 284
donut 328 cake 310 chair 1771
couch 261 potted plant 342 bed 163
dining table 695 toilet 179 tv 288
laptop 231 mouse 106 remote 283
keyboard 153 cell phone 262 microwave 55
oven 143 toaster 9 sink 225
refrigerator 126 book 1129 clock 267
vase 274 scissors 36 teddy bear 190
hair drier 11 toothbrush 57
total 36335 

[04/14 09:16:58] d2.data.common INFO: Serializing 5000 elements to byte tensors and concatenating them all ... [04/14 09:16:59] d2.data.common INFO: Serialized dataset takes 19.20 MiB [04/14 09:16:59] d2.evaluation.evaluator INFO: Start inference on 5000 images [04/14 09:17:02] d2.evaluation.evaluator INFO: Inference done 11/5000. 0.1866 s / img. ETA=0:15:41 [04/14 09:17:07] d2.evaluation.evaluator INFO: Inference done 38/5000. 0.1852 s / img. ETA=0:15:34 [04/14 09:17:12] d2.evaluation.evaluator INFO: Inference done 64/5000. 0.1876 s / img. ETA=0:15:41 [04/14 09:17:17] d2.evaluation.evaluator INFO: Inference done 91/5000. 0.1873 s / img. ETA=0:15:35 [04/14 09:17:22] d2.evaluation.evaluator INFO: Inference done 119/5000. 0.1861 s / img. ETA=0:15:23 [04/14 09:17:27] d2.evaluation.evaluator INFO: Inference done 145/5000. 0.1871 s / img. ETA=0:15:23 [04/14 09:17:32] d2.evaluation.evaluator INFO: Inference done 172/5000. 0.1871 s / img. ETA=0:15:18 [04/14 09:17:37] d2.evaluation.evaluator INFO: Inference done 199/5000. 0.1871 s / img. ETA=0:15:13 [04/14 09:17:42] d2.evaluation.evaluator INFO: Inference done 226/5000. 0.1866 s / img. ETA=0:15:05 [04/14 09:17:47] d2.evaluation.evaluator INFO: Inference done 252/5000. 0.1870 s / img. ETA=0:15:02 [04/14 09:17:52] d2.evaluation.evaluator INFO: Inference done 279/5000. 0.1866 s / img. ETA=0:14:55 [04/14 09:17:57] d2.evaluation.evaluator INFO: Inference done 305/5000. 0.1869 s / img. ETA=0:14:51 [04/14 09:18:03] d2.evaluation.evaluator INFO: Inference done 332/5000. 0.1869 s / img. ETA=0:14:46 [04/14 09:18:08] d2.evaluation.evaluator INFO: Inference done 358/5000. 0.1873 s / img. ETA=0:14:43 [04/14 09:18:13] d2.evaluation.evaluator INFO: Inference done 385/5000. 0.1871 s / img. ETA=0:14:37 [04/14 09:18:18] d2.evaluation.evaluator INFO: Inference done 412/5000. 0.1869 s / img. ETA=0:14:31 [04/14 09:18:23] d2.evaluation.evaluator INFO: Inference done 439/5000. 0.1869 s / img. ETA=0:14:26 [04/14 09:18:28] d2.evaluation.evaluator INFO: Inference done 466/5000. 0.1871 s / img. ETA=0:14:22 [04/14 09:18:33] d2.evaluation.evaluator INFO: Inference done 492/5000. 0.1874 s / img. ETA=0:14:18 [04/14 09:18:38] d2.evaluation.evaluator INFO: Inference done 520/5000. 0.1870 s / img. ETA=0:14:11 [04/14 09:18:43] d2.evaluation.evaluator INFO: Inference done 547/5000. 0.1869 s / img. ETA=0:14:06 [04/14 09:18:49] d2.evaluation.evaluator INFO: Inference done 574/5000. 0.1870 s / img. ETA=0:14:01 [04/14 09:18:54] d2.evaluation.evaluator INFO: Inference done 601/5000. 0.1869 s / img. ETA=0:13:55 [04/14 09:18:59] d2.evaluation.evaluator INFO: Inference done 627/5000. 0.1870 s / img. ETA=0:13:51 [04/14 09:19:04] d2.evaluation.evaluator INFO: Inference done 654/5000. 0.1869 s / img. ETA=0:13:45 [04/14 09:19:09] d2.evaluation.evaluator INFO: Inference done 680/5000. 0.1870 s / img. ETA=0:13:41 [04/14 09:19:14] d2.evaluation.evaluator INFO: Inference done 707/5000. 0.1871 s / img. ETA=0:13:36 [04/14 09:19:19] d2.evaluation.evaluator INFO: Inference done 733/5000. 0.1872 s / img. ETA=0:13:31 [04/14 09:19:24] d2.evaluation.evaluator INFO: Inference done 759/5000. 0.1873 s / img. ETA=0:13:27 [04/14 09:19:29] d2.evaluation.evaluator INFO: Inference done 786/5000. 0.1874 s / img. ETA=0:13:22 [04/14 09:19:34] d2.evaluation.evaluator INFO: Inference done 813/5000. 0.1874 s / img. ETA=0:13:17 [04/14 09:19:39] d2.evaluation.evaluator INFO: Inference done 840/5000. 0.1873 s / img. ETA=0:13:12 [04/14 09:19:44] d2.evaluation.evaluator INFO: Inference done 866/5000. 0.1874 s / img. ETA=0:13:07 [04/14 09:19:50] d2.evaluation.evaluator INFO: Inference done 893/5000. 0.1874 s / img. ETA=0:13:02 [04/14 09:19:55] d2.evaluation.evaluator INFO: Inference done 920/5000. 0.1873 s / img. ETA=0:12:56 [04/14 09:20:00] d2.evaluation.evaluator INFO: Inference done 947/5000. 0.1873 s / img. ETA=0:12:51 [04/14 09:20:05] d2.evaluation.evaluator INFO: Inference done 974/5000. 0.1873 s / img. ETA=0:12:46 [04/14 09:20:10] d2.evaluation.evaluator INFO: Inference done 1001/5000. 0.1873 s / img. ETA=0:12:41 [04/14 09:20:15] d2.evaluation.evaluator INFO: Inference done 1028/5000. 0.1872 s / img. ETA=0:12:35 [04/14 09:20:20] d2.evaluation.evaluator INFO: Inference done 1055/5000. 0.1872 s / img. ETA=0:12:30 [04/14 09:20:25] d2.evaluation.evaluator INFO: Inference done 1081/5000. 0.1873 s / img. ETA=0:12:26 [04/14 09:20:30] d2.evaluation.evaluator INFO: Inference done 1108/5000. 0.1874 s / img. ETA=0:12:21 [04/14 09:20:36] d2.evaluation.evaluator INFO: Inference done 1134/5000. 0.1875 s / img. ETA=0:12:16 [04/14 09:20:41] d2.evaluation.evaluator INFO: Inference done 1161/5000. 0.1874 s / img. ETA=0:12:11 [04/14 09:20:46] d2.evaluation.evaluator INFO: Inference done 1187/5000. 0.1875 s / img. ETA=0:12:06 [04/14 09:20:51] d2.evaluation.evaluator INFO: Inference done 1213/5000. 0.1876 s / img. ETA=0:12:01 [04/14 09:20:56] d2.evaluation.evaluator INFO: Inference done 1240/5000. 0.1875 s / img. ETA=0:11:56 [04/14 09:21:01] d2.evaluation.evaluator INFO: Inference done 1267/5000. 0.1874 s / img. ETA=0:11:51 [04/14 09:21:06] d2.evaluation.evaluator INFO: Inference done 1293/5000. 0.1875 s / img. ETA=0:11:46 [04/14 09:21:11] d2.evaluation.evaluator INFO: Inference done 1321/5000. 0.1873 s / img. ETA=0:11:40 [04/14 09:21:16] d2.evaluation.evaluator INFO: Inference done 1348/5000. 0.1874 s / img. ETA=0:11:35 [04/14 09:21:21] d2.evaluation.evaluator INFO: Inference done 1375/5000. 0.1873 s / img. ETA=0:11:30 [04/14 09:21:26] d2.evaluation.evaluator INFO: Inference done 1402/5000. 0.1873 s / img. ETA=0:11:24 [04/14 09:21:32] d2.evaluation.evaluator INFO: Inference done 1430/5000. 0.1872 s / img. ETA=0:11:19 [04/14 09:21:37] d2.evaluation.evaluator INFO: Inference done 1456/5000. 0.1872 s / img. ETA=0:11:14 [04/14 09:21:42] d2.evaluation.evaluator INFO: Inference done 1484/5000. 0.1871 s / img. ETA=0:11:08 [04/14 09:21:47] d2.evaluation.evaluator INFO: Inference done 1512/5000. 0.1870 s / img. ETA=0:11:02 [04/14 09:21:52] d2.evaluation.evaluator INFO: Inference done 1539/5000. 0.1870 s / img. ETA=0:10:57 [04/14 09:21:57] d2.evaluation.evaluator INFO: Inference done 1565/5000. 0.1870 s / img. ETA=0:10:52 [04/14 09:22:02] d2.evaluation.evaluator INFO: Inference done 1591/5000. 0.1871 s / img. ETA=0:10:48 [04/14 09:22:07] d2.evaluation.evaluator INFO: Inference done 1618/5000. 0.1871 s / img. ETA=0:10:42 [04/14 09:22:12] d2.evaluation.evaluator INFO: Inference done 1645/5000. 0.1871 s / img. ETA=0:10:37 [04/14 09:22:17] d2.evaluation.evaluator INFO: Inference done 1672/5000. 0.1870 s / img. ETA=0:10:32 [04/14 09:22:22] d2.evaluation.evaluator INFO: Inference done 1698/5000. 0.1871 s / img. ETA=0:10:27 [04/14 09:22:27] d2.evaluation.evaluator INFO: Inference done 1725/5000. 0.1871 s / img. ETA=0:10:22 [04/14 09:22:33] d2.evaluation.evaluator INFO: Inference done 1752/5000. 0.1871 s / img. ETA=0:10:17 [04/14 09:22:38] d2.evaluation.evaluator INFO: Inference done 1779/5000. 0.1870 s / img. ETA=0:10:12 [04/14 09:22:43] d2.evaluation.evaluator INFO: Inference done 1806/5000. 0.1870 s / img. ETA=0:10:06 [04/14 09:22:48] d2.evaluation.evaluator INFO: Inference done 1833/5000. 0.1870 s / img. ETA=0:10:01 [04/14 09:22:53] d2.evaluation.evaluator INFO: Inference done 1860/5000. 0.1869 s / img. ETA=0:09:56 [04/14 09:22:58] d2.evaluation.evaluator INFO: Inference done 1886/5000. 0.1870 s / img. ETA=0:09:51 [04/14 09:23:03] d2.evaluation.evaluator INFO: Inference done 1912/5000. 0.1870 s / img. ETA=0:09:46 [04/14 09:23:08] d2.evaluation.evaluator INFO: Inference done 1939/5000. 0.1870 s / img. ETA=0:09:41 [04/14 09:23:13] d2.evaluation.evaluator INFO: Inference done 1966/5000. 0.1870 s / img. ETA=0:09:36 [04/14 09:23:18] d2.evaluation.evaluator INFO: Inference done 1993/5000. 0.1870 s / img. ETA=0:09:31 [04/14 09:23:23] d2.evaluation.evaluator INFO: Inference done 2019/5000. 0.1870 s / img. ETA=0:09:26 [04/14 09:23:28] d2.evaluation.evaluator INFO: Inference done 2046/5000. 0.1870 s / img. ETA=0:09:21 [04/14 09:23:33] d2.evaluation.evaluator INFO: Inference done 2073/5000. 0.1870 s / img. ETA=0:09:16 [04/14 09:23:38] d2.evaluation.evaluator INFO: Inference done 2100/5000. 0.1869 s / img. ETA=0:09:10 [04/14 09:23:44] d2.evaluation.evaluator INFO: Inference done 2127/5000. 0.1869 s / img. ETA=0:09:05 [04/14 09:23:49] d2.evaluation.evaluator INFO: Inference done 2154/5000. 0.1869 s / img. ETA=0:09:00 [04/14 09:23:54] d2.evaluation.evaluator INFO: Inference done 2181/5000. 0.1869 s / img. ETA=0:08:55 [04/14 09:23:59] d2.evaluation.evaluator INFO: Inference done 2207/5000. 0.1869 s / img. ETA=0:08:50 [04/14 09:24:04] d2.evaluation.evaluator INFO: Inference done 2234/5000. 0.1869 s / img. ETA=0:08:45 [04/14 09:24:09] d2.evaluation.evaluator INFO: Inference done 2260/5000. 0.1870 s / img. ETA=0:08:40 [04/14 09:24:14] d2.evaluation.evaluator INFO: Inference done 2287/5000. 0.1870 s / img. ETA=0:08:35 [04/14 09:24:19] d2.evaluation.evaluator INFO: Inference done 2314/5000. 0.1869 s / img. ETA=0:08:30 [04/14 09:24:24] d2.evaluation.evaluator INFO: Inference done 2341/5000. 0.1869 s / img. ETA=0:08:25 [04/14 09:24:29] d2.evaluation.evaluator INFO: Inference done 2367/5000. 0.1870 s / img. ETA=0:08:20 [04/14 09:24:34] d2.evaluation.evaluator INFO: Inference done 2394/5000. 0.1870 s / img. ETA=0:08:15 [04/14 09:24:40] d2.evaluation.evaluator INFO: Inference done 2421/5000. 0.1870 s / img. ETA=0:08:10 [04/14 09:24:45] d2.evaluation.evaluator INFO: Inference done 2447/5000. 0.1871 s / img. ETA=0:08:05 [04/14 09:24:50] d2.evaluation.evaluator INFO: Inference done 2474/5000. 0.1870 s / img. ETA=0:08:00 [04/14 09:24:55] d2.evaluation.evaluator INFO: Inference done 2501/5000. 0.1870 s / img. ETA=0:07:54 [04/14 09:25:00] d2.evaluation.evaluator INFO: Inference done 2528/5000. 0.1870 s / img. ETA=0:07:49 [04/14 09:25:05] d2.evaluation.evaluator INFO: Inference done 2555/5000. 0.1870 s / img. ETA=0:07:44 [04/14 09:25:10] d2.evaluation.evaluator INFO: Inference done 2583/5000. 0.1869 s / img. ETA=0:07:39 [04/14 09:25:15] d2.evaluation.evaluator INFO: Inference done 2610/5000. 0.1869 s / img. ETA=0:07:33 [04/14 09:25:20] d2.evaluation.evaluator INFO: Inference done 2636/5000. 0.1870 s / img. ETA=0:07:29 [04/14 09:25:25] d2.evaluation.evaluator INFO: Inference done 2663/5000. 0.1870 s / img. ETA=0:07:23 [04/14 09:25:30] d2.evaluation.evaluator INFO: Inference done 2689/5000. 0.1870 s / img. ETA=0:07:19 [04/14 09:25:36] d2.evaluation.evaluator INFO: Inference done 2716/5000. 0.1870 s / img. ETA=0:07:14 [04/14 09:25:41] d2.evaluation.evaluator INFO: Inference done 2743/5000. 0.1870 s / img. ETA=0:07:08 [04/14 09:25:46] d2.evaluation.evaluator INFO: Inference done 2770/5000. 0.1870 s / img. ETA=0:07:03 [04/14 09:25:51] d2.evaluation.evaluator INFO: Inference done 2797/5000. 0.1869 s / img. ETA=0:06:58 [04/14 09:25:56] d2.evaluation.evaluator INFO: Inference done 2824/5000. 0.1869 s / img. ETA=0:06:53 [04/14 09:26:01] d2.evaluation.evaluator INFO: Inference done 2850/5000. 0.1870 s / img. ETA=0:06:48 [04/14 09:26:06] d2.evaluation.evaluator INFO: Inference done 2877/5000. 0.1870 s / img. ETA=0:06:43 [04/14 09:26:11] d2.evaluation.evaluator INFO: Inference done 2904/5000. 0.1870 s / img. ETA=0:06:38 [04/14 09:26:16] d2.evaluation.evaluator INFO: Inference done 2931/5000. 0.1870 s / img. ETA=0:06:33 [04/14 09:26:22] d2.evaluation.evaluator INFO: Inference done 2958/5000. 0.1870 s / img. ETA=0:06:27 [04/14 09:26:27] d2.evaluation.evaluator INFO: Inference done 2985/5000. 0.1870 s / img. ETA=0:06:22 [04/14 09:26:32] d2.evaluation.evaluator INFO: Inference done 3012/5000. 0.1870 s / img. ETA=0:06:17 [04/14 09:26:37] d2.evaluation.evaluator INFO: Inference done 3038/5000. 0.1870 s / img. ETA=0:06:12 [04/14 09:26:42] d2.evaluation.evaluator INFO: Inference done 3064/5000. 0.1871 s / img. ETA=0:06:08 [04/14 09:26:47] d2.evaluation.evaluator INFO: Inference done 3090/5000. 0.1871 s / img. ETA=0:06:03 [04/14 09:26:52] d2.evaluation.evaluator INFO: Inference done 3117/5000. 0.1871 s / img. ETA=0:05:57 [04/14 09:26:57] d2.evaluation.evaluator INFO: Inference done 3143/5000. 0.1871 s / img. ETA=0:05:53 [04/14 09:27:02] d2.evaluation.evaluator INFO: Inference done 3170/5000. 0.1871 s / img. ETA=0:05:47 [04/14 09:27:07] d2.evaluation.evaluator INFO: Inference done 3197/5000. 0.1871 s / img. ETA=0:05:42 [04/14 09:27:12] d2.evaluation.evaluator INFO: Inference done 3224/5000. 0.1871 s / img. ETA=0:05:37 [04/14 09:27:17] d2.evaluation.evaluator INFO: Inference done 3251/5000. 0.1871 s / img. ETA=0:05:32 [04/14 09:27:23] d2.evaluation.evaluator INFO: Inference done 3278/5000. 0.1871 s / img. ETA=0:05:27 [04/14 09:27:28] d2.evaluation.evaluator INFO: Inference done 3305/5000. 0.1871 s / img. ETA=0:05:22 [04/14 09:27:33] d2.evaluation.evaluator INFO: Inference done 3332/5000. 0.1871 s / img. ETA=0:05:17 [04/14 09:27:38] d2.evaluation.evaluator INFO: Inference done 3359/5000. 0.1871 s / img. ETA=0:05:11 [04/14 09:27:43] d2.evaluation.evaluator INFO: Inference done 3386/5000. 0.1871 s / img. ETA=0:05:06 [04/14 09:27:48] d2.evaluation.evaluator INFO: Inference done 3414/5000. 0.1870 s / img. ETA=0:05:01 [04/14 09:27:53] d2.evaluation.evaluator INFO: Inference done 3440/5000. 0.1871 s / img. ETA=0:04:56 [04/14 09:27:58] d2.evaluation.evaluator INFO: Inference done 3466/5000. 0.1871 s / img. ETA=0:04:51 [04/14 09:28:04] d2.evaluation.evaluator INFO: Inference done 3494/5000. 0.1871 s / img. ETA=0:04:46 [04/14 09:28:09] d2.evaluation.evaluator INFO: Inference done 3521/5000. 0.1870 s / img. ETA=0:04:41 [04/14 09:28:14] d2.evaluation.evaluator INFO: Inference done 3548/5000. 0.1870 s / img. ETA=0:04:35 [04/14 09:28:19] d2.evaluation.evaluator INFO: Inference done 3575/5000. 0.1870 s / img. ETA=0:04:30 [04/14 09:28:24] d2.evaluation.evaluator INFO: Inference done 3602/5000. 0.1870 s / img. ETA=0:04:25 [04/14 09:28:29] d2.evaluation.evaluator INFO: Inference done 3629/5000. 0.1870 s / img. ETA=0:04:20 [04/14 09:28:34] d2.evaluation.evaluator INFO: Inference done 3655/5000. 0.1870 s / img. ETA=0:04:15 [04/14 09:28:39] d2.evaluation.evaluator INFO: Inference done 3682/5000. 0.1870 s / img. ETA=0:04:10 [04/14 09:28:44] d2.evaluation.evaluator INFO: Inference done 3709/5000. 0.1870 s / img. ETA=0:04:05 [04/14 09:28:49] d2.evaluation.evaluator INFO: Inference done 3735/5000. 0.1870 s / img. ETA=0:04:00 [04/14 09:28:54] d2.evaluation.evaluator INFO: Inference done 3762/5000. 0.1870 s / img. ETA=0:03:55 [04/14 09:29:00] d2.evaluation.evaluator INFO: Inference done 3789/5000. 0.1870 s / img. ETA=0:03:50 [04/14 09:29:05] d2.evaluation.evaluator INFO: Inference done 3816/5000. 0.1870 s / img. ETA=0:03:44 [04/14 09:29:10] d2.evaluation.evaluator INFO: Inference done 3842/5000. 0.1871 s / img. ETA=0:03:40 [04/14 09:29:15] d2.evaluation.evaluator INFO: Inference done 3869/5000. 0.1871 s / img. ETA=0:03:34 [04/14 09:29:20] d2.evaluation.evaluator INFO: Inference done 3896/5000. 0.1870 s / img. ETA=0:03:29 [04/14 09:29:25] d2.evaluation.evaluator INFO: Inference done 3923/5000. 0.1870 s / img. ETA=0:03:24 [04/14 09:29:30] d2.evaluation.evaluator INFO: Inference done 3950/5000. 0.1870 s / img. ETA=0:03:19 [04/14 09:29:35] d2.evaluation.evaluator INFO: Inference done 3977/5000. 0.1870 s / img. ETA=0:03:14 [04/14 09:29:40] d2.evaluation.evaluator INFO: Inference done 4005/5000. 0.1870 s / img. ETA=0:03:08 [04/14 09:29:45] d2.evaluation.evaluator INFO: Inference done 4032/5000. 0.1870 s / img. ETA=0:03:03 [04/14 09:29:50] d2.evaluation.evaluator INFO: Inference done 4059/5000. 0.1870 s / img. ETA=0:02:58 [04/14 09:29:56] d2.evaluation.evaluator INFO: Inference done 4086/5000. 0.1869 s / img. ETA=0:02:53 [04/14 09:30:01] d2.evaluation.evaluator INFO: Inference done 4112/5000. 0.1870 s / img. ETA=0:02:48 [04/14 09:30:06] d2.evaluation.evaluator INFO: Inference done 4138/5000. 0.1870 s / img. ETA=0:02:43 [04/14 09:30:11] d2.evaluation.evaluator INFO: Inference done 4165/5000. 0.1870 s / img. ETA=0:02:38 [04/14 09:30:16] d2.evaluation.evaluator INFO: Inference done 4192/5000. 0.1870 s / img. ETA=0:02:33 [04/14 09:30:21] d2.evaluation.evaluator INFO: Inference done 4219/5000. 0.1870 s / img. ETA=0:02:28 [04/14 09:30:26] d2.evaluation.evaluator INFO: Inference done 4247/5000. 0.1870 s / img. ETA=0:02:23 [04/14 09:30:31] d2.evaluation.evaluator INFO: Inference done 4274/5000. 0.1870 s / img. ETA=0:02:17 [04/14 09:30:37] d2.evaluation.evaluator INFO: Inference done 4302/5000. 0.1869 s / img. ETA=0:02:12 [04/14 09:30:42] d2.evaluation.evaluator INFO: Inference done 4329/5000. 0.1869 s / img. ETA=0:02:07 [04/14 09:30:47] d2.evaluation.evaluator INFO: Inference done 4356/5000. 0.1869 s / img. ETA=0:02:02 [04/14 09:30:52] d2.evaluation.evaluator INFO: Inference done 4384/5000. 0.1869 s / img. ETA=0:01:56 [04/14 09:30:57] d2.evaluation.evaluator INFO: Inference done 4411/5000. 0.1869 s / img. ETA=0:01:51 [04/14 09:31:02] d2.evaluation.evaluator INFO: Inference done 4439/5000. 0.1869 s / img. ETA=0:01:46 [04/14 09:31:07] d2.evaluation.evaluator INFO: Inference done 4465/5000. 0.1869 s / img. ETA=0:01:41 [04/14 09:31:12] d2.evaluation.evaluator INFO: Inference done 4492/5000. 0.1869 s / img. ETA=0:01:36 [04/14 09:31:17] d2.evaluation.evaluator INFO: Inference done 4519/5000. 0.1869 s / img. ETA=0:01:31 [04/14 09:31:23] d2.evaluation.evaluator INFO: Inference done 4547/5000. 0.1868 s / img. ETA=0:01:25 [04/14 09:31:28] d2.evaluation.evaluator INFO: Inference done 4574/5000. 0.1869 s / img. ETA=0:01:20 [04/14 09:31:33] d2.evaluation.evaluator INFO: Inference done 4600/5000. 0.1869 s / img. ETA=0:01:15 [04/14 09:31:38] d2.evaluation.evaluator INFO: Inference done 4627/5000. 0.1869 s / img. ETA=0:01:10 [04/14 09:31:43] d2.evaluation.evaluator INFO: Inference done 4654/5000. 0.1869 s / img. ETA=0:01:05 [04/14 09:31:48] d2.evaluation.evaluator INFO: Inference done 4682/5000. 0.1868 s / img. ETA=0:01:00 [04/14 09:31:53] d2.evaluation.evaluator INFO: Inference done 4709/5000. 0.1868 s / img. ETA=0:00:55 [04/14 09:31:58] d2.evaluation.evaluator INFO: Inference done 4736/5000. 0.1868 s / img. ETA=0:00:50 [04/14 09:32:04] d2.evaluation.evaluator INFO: Inference done 4763/5000. 0.1869 s / img. ETA=0:00:44 [04/14 09:32:09] d2.evaluation.evaluator INFO: Inference done 4790/5000. 0.1869 s / img. ETA=0:00:39 [04/14 09:32:14] d2.evaluation.evaluator INFO: Inference done 4817/5000. 0.1869 s / img. ETA=0:00:34 [04/14 09:32:19] d2.evaluation.evaluator INFO: Inference done 4845/5000. 0.1868 s / img. ETA=0:00:29 [04/14 09:32:24] d2.evaluation.evaluator INFO: Inference done 4871/5000. 0.1868 s / img. ETA=0:00:24 [04/14 09:32:29] d2.evaluation.evaluator INFO: Inference done 4897/5000. 0.1869 s / img. ETA=0:00:19 [04/14 09:32:34] d2.evaluation.evaluator INFO: Inference done 4924/5000. 0.1869 s / img. ETA=0:00:14 [04/14 09:32:39] d2.evaluation.evaluator INFO: Inference done 4950/5000. 0.1869 s / img. ETA=0:00:09 [04/14 09:32:45] d2.evaluation.evaluator INFO: Inference done 4977/5000. 0.1869 s / img. ETA=0:00:04 [04/14 09:32:49] d2.evaluation.evaluator INFO: Total inference time: 0:15:48.578322 (0.189906 s / img per device, on 1 devices) [04/14 09:32:49] d2.evaluation.evaluator INFO: Total inference pure compute time: 0:15:33 (0.186905 s / img per device, on 1 devices) [04/14 09:32:49] d2.evaluation.coco_evaluation INFO: Preparing results for COCO format ... [04/14 09:32:49] d2.evaluation.coco_evaluation INFO: Saving results to /inference/coco_instances_results.json [04/14 09:32:49] d2.evaluation.coco_evaluation INFO: Evaluating predictions ... [04/14 09:32:49] d2.evaluation.coco_evaluation WARNING: No predictions from the model! [04/14 09:32:49] d2.evaluation.testing INFO: copypaste: Task: bbox [04/14 09:32:49] d2.evaluation.testing INFO: copypaste: AP,AP50,AP75,APs,APm,APl [04/14 09:32:49] d2.evaluation.testing INFO: copypaste: nan,nan,nan,nan,nan,nan [04/14 09:32:49] d2.utils.events INFO: eta: 0:00:00 iter: 999 total_loss: 8.890 loss_cls: 2.370 loss_box_reg: 0.956 loss_query: 5.589 time: 0.8127 data_time: 0.0184 lr: 0.009990 max_mem: 28515M [04/14 09:32:49] d2.engine.hooks INFO: Overall training speed: 997 iterations in 0:13:31 (0.8135 s / it) [04/14 09:32:49] d2.engine.hooks INFO: Total training time: 0:29:23 (0:15:52 on hooks)

Hi. This problem is weird. Can you please copy-paste the last part of the training log?

XuKer commented 2 years ago

Hi. This problem is weird. Can you please copy-paste the last part of the training log?

thanks a lot for your answer

ChenhongyiYang commented 2 years ago

Hi, I find the model is only trained for 1000 iterations, which is far from a good performance. Can you please train it with the full schedule (90000 iterations with batch size 16) and have a try?