search

lyuwenyu / RT-DETR

[CVPR 2024] Official RT-DETR (RTDETR paddle pytorch), Real-Time DEtection TRansformer, DETRs Beat YOLOs on Real-time Object Detection. 🔥 🔥 🔥
Apache License 2.0
2.21k stars 242 forks source link

求助，关于torch版本预训练模型的测试和微调 #81

Open bartbuaa opened 11 months ago

bartbuaa commented 11 months ago

Hi作者您好： Rt-DETR是非常引人瞩目的成果，在TensorRT环境有非常显著的耗时改善； 我在尝试复现您的Torch代码时，发现测试时加载rtdetr_r50vd_6x_coco_from_paddle.pth时验证结果正常，Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ]=0.531和 model zoo的结果一致； 然而，当在训练中加载模型后，log似乎出现了loss过大的情况，由于训练和测试resume的代码是同一个，所以我比较确信加载模型是没有问题的，我的训练log如下： Epoch: [0] [ 100/7392] eta: 1:58:50 lr: 0.000100 loss: 39.5110 (40.6936) loss_vfl: 0.6387 (0.5253) loss_bbox: 1.0808 (1.2656) loss_giou: 1.5173 (1.6033) loss_vfl_aux_0: 0.6074 (0.4685) loss_bbox_aux_0: 1.1071 (1.3050) loss_giou_aux_0: 1.5485 (1.6285) loss_vfl_aux_1: 0.6030 (0.4892) loss_bbox_aux_1: 1.0991 (1.2872) loss_giou_aux_1: 1.5320 (1.6203) loss_vfl_aux_2: 0.6250 (0.5054) loss_bbox_aux_2: 1.0899 (1.2781) loss_giou_aux_2: 1.5273 (1.6120) loss_vfl_aux_3: 0.6367 (0.5171) loss_bbox_aux_3: 1.0931 (1.2726) loss_giou_aux_3: 1.5076 (1.6085) loss_vfl_aux_4: 0.6333 (0.5200) loss_bbox_aux_4: 1.0872 (1.2672) loss_giou_aux_4: 1.5231 (1.6074) loss_vfl_aux_5: 0.5547 (0.4435) loss_bbox_aux_5: 1.1503 (1.3328) loss_giou_aux_5: 1.5580 (1.6498) loss_vfl_dn_0: 0.5308 (0.6228) loss_bbox_dn_0: 0.8397 (0.8256) loss_giou_dn_0: 1.3402 (1.3400) loss_vfl_dn_1: 0.5400 (0.6253) loss_bbox_dn_1: 0.8400 (0.8329) loss_giou_dn_1: 1.3398 (1.3411) loss_vfl_dn_2: 0.5493 (0.6232) loss_bbox_dn_2: 0.8390 (0.8400) loss_giou_dn_2: 1.3391 (1.3448) loss_vfl_dn_3: 0.5532 (0.6290) loss_bbox_dn_3: 0.8385 (0.8464) loss_giou_dn_3: 1.3398 (1.3485) loss_vfl_dn_4: 0.5625 (0.6270) loss_bbox_dn_4: 0.8380 (0.8514) loss_giou_dn_4: 1.3411 (1.3523) loss_vfl_dn_5: 0.5625 (0.6219) loss_bbox_dn_5: 0.8376 (0.8570) loss_giou_dn_5: 1.3423 (1.3567) time: 0.9180 data: 0.0465 max mem: 22198 Epoch: [0] [ 200/7392] eta: 1:50:17 lr: 0.000100 loss: 38.0543 (39.7114) loss_vfl: 0.6357 (0.6036) loss_bbox: 0.9486 (1.1422) loss_giou: 1.5369 (1.5641) loss_vfl_aux_0: 0.5776 (0.5436) loss_bbox_aux_0: 0.9718 (1.1718) loss_giou_aux_0: 1.5599 (1.5804) loss_vfl_aux_1: 0.5991 (0.5635) loss_bbox_aux_1: 0.9795 (1.1608) loss_giou_aux_1: 1.5483 (1.5739) loss_vfl_aux_2: 0.6001 (0.5805) loss_bbox_aux_2: 0.9636 (1.1528) loss_giou_aux_2: 1.5446 (1.5691) loss_vfl_aux_3: 0.6060 (0.5916) loss_bbox_aux_3: 0.9589 (1.1486) loss_giou_aux_3: 1.5398 (1.5663) loss_vfl_aux_4: 0.6226 (0.5961) loss_bbox_aux_4: 0.9484 (1.1443) loss_giou_aux_4: 1.5389 (1.5659) loss_vfl_aux_5: 0.5415 (0.5124) loss_bbox_aux_5: 0.9901 (1.1955) loss_giou_aux_5: 1.5683 (1.5981) loss_vfl_dn_0: 0.4453 (0.5500) loss_bbox_dn_0: 0.7753 (0.8410) loss_giou_dn_0: 1.3354 (1.3383) loss_vfl_dn_1: 0.4578 (0.5585) loss_bbox_dn_1: 0.7749 (0.8444) loss_giou_dn_1: 1.3277 (1.3392) loss_vfl_dn_2: 0.4792 (0.5706) loss_bbox_dn_2: 0.7736 (0.8475) loss_giou_dn_2: 1.3347 (1.3425) loss_vfl_dn_3: 0.5005 (0.5804) loss_bbox_dn_3: 0.7729 (0.8505) loss_giou_dn_3: 1.3367 (1.3454) loss_vfl_dn_4: 0.5215 (0.5857) loss_bbox_dn_4: 0.7723 (0.8528) loss_giou_dn_4: 1.3369 (1.3482) loss_vfl_dn_5: 0.5215 (0.5845) loss_bbox_dn_5: 0.7721 (0.8555) loss_giou_dn_5: 1.3372 (1.3513) time: 0.8681 data: 0.0457 max mem: 22201 上述log似乎和您在paddle环境中的不太一致，我相信加载了0.531的预训练模型后，loss应该不会有这么高，不知可否提供一些帮助，非常感谢！

paddle log： [04/02 08:10:44] ppdet.engine INFO: Epoch: [71] [7100/7329] learning_rate: 0.000100 loss_class: 0.560868 loss_bbox: 0.130616 loss_giou: 0.407824 loss_class_aux: 3.533982 loss_bbox_aux: 0.884162 loss_giou_aux: 2.528244 loss_class_dn: 0.341166 loss_bbox_dn: 0.131751 loss_giou_dn: 0.349444 loss_class_aux_dn: 1.835036 loss_bbox_aux_dn: 0.793820 loss_giou_aux_dn: 2.015112 loss: 13.715096 eta: 0:02:05 batch_cost: 0.5242 data_cost: 0.0014 ips: 7.6313 images/s [04/02 08:11:44] ppdet.engine INFO: Epoch: [71] [7200/7329] learning_rate: 0.000100 loss_class: 0.602725 loss_bbox: 0.130830 loss_giou: 0.406435 loss_class_aux: 3.799356 loss_bbox_aux: 0.888293 loss_giou_aux: 2.604114 loss_class_dn: 0.336983 loss_bbox_dn: 0.131602 loss_giou_dn: 0.352628 loss_class_aux_dn: 1.860950 loss_bbox_aux_dn: 0.777279 loss_giou_aux_dn: 2.050699 loss: 14.956850 eta: 0:01:10 batch_cost: 0.5429 data_cost: 0.0013 ips: 7.3683 images/s

lyuwenyu commented 11 months ago

你是在那个位置load的torch的参数 这里有个问题是torch目前没有ema的参数

bartbuaa commented 11 months ago

你好，感谢回复！ 训练时加载模型的位置在：src/solver/det_solver/fit/self.train()的line44 if self.cfg.resume: print(f'Resume checkpoint from {self.cfg.resume}') self.resume(self.cfg.resume) 在训练log中显示如下，确实有ema参数的显示 Start training Initial lr: [0.0001, 0.0001, 0.0001, 0.0001] resume from Model/rtdetr_r50vd_6x_coco_from_paddle.pth Loading ema.state_dict loading annotations into memory... Done (t=15.20s) Epoch: [0] [ 100/7392] eta: 1:58:50 lr: 0.000100 loss: 39.5110 (40.6936) loss_vfl: 0.6387 (0.5253) loss_bbox: 1.0808 (1.2656) loss_giou: 1.5173 (1.6033) loss_vfl_aux_0: 0.6074 (0.4685) loss_bbox_aux_0: 1.1071 (1.3050) loss_giou_aux_0: 1.5485 (1.6285) loss_vfl_aux_1: 0.6030 (0.4892) loss_bbox_aux_1: 1.0991 (1.2872) loss_giou_aux_1: 1.5320 (1.6203) loss_vfl_aux_2: 0.6250 (0.5054) loss_bbox_aux_2: 1.0899 (1.2781) loss_giou_aux_2: 1.5273 (1.6120) loss_vfl_aux_3: 0.6367 (0.5171) loss_bbox_aux_3: 1.0931 (1.2726) loss_giou_aux_3: 1.5076 (1.6085) loss_vfl_aux_4: 0.6333 (0.5200) loss_bbox_aux_4: 1.0872 (1.2672) loss_giou_aux_4: 1.5231 (1.6074) loss_vfl_aux_5: 0.5547 (0.4435) loss_bbox_aux_5: 1.1503 (1.3328) loss_giou_aux_5: 1.5580 (1.6498) loss_vfl_dn_0: 0.5308 (0.6228) loss_bbox_dn_0: 0.8397 (0.8256) loss_giou_dn_0: 1.3402 (1.3400) loss_vfl_dn_1: 0.5400 (0.6253) loss_bbox_dn_1: 0.8400 (0.8329) loss_giou_dn_1: 1.3398 (1.3411) loss_vfl_dn_2: 0.5493 (0.6232) loss_bbox_dn_2: 0.8390 (0.8400) loss_giou_dn_2: 1.3391 (1.3448) loss_vfl_dn_3: 0.5532 (0.6290) loss_bbox_dn_3: 0.8385 (0.8464) loss_giou_dn_3: 1.3398 (1.3485) loss_vfl_dn_4: 0.5625 (0.6270) loss_bbox_dn_4: 0.8380 (0.8514) loss_giou_dn_4: 1.3411 (1.3523) loss_vfl_dn_5: 0.5625 (0.6219) loss_bbox_dn_5: 0.8376 (0.8570) loss_giou_dn_5: 1.3423 (1.3567) time: 0.9180 data: 0.0465 max mem: 22198

测试时加载模型的位置在：src/solver/det_solver/fit/self.eval()的line59 ，使用的也是self.resume() 测试时log为 loading annotations into memory... Done (t=0.66s) creating index... index created! resume from Model/rtdetr_r50vd_6x_coco_from_paddle.pth Loading ema.state_dict 但预测的结果正常 Averaged stats: Accumulating evaluation results... DONE (t=25.03s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.531 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.712 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.577 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.347 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.577 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.701 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.390 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.655 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.721 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.547 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.765 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.880

lyuwenyu commented 11 months ago

这个逻辑不对，， 我转的参数 你看下key 只有ema的key 用resume会导致这里只加载ema的state_dict

fintune的话需要把ema的参数写到model里去 你可以在这个位置加一下这个逻辑

lyuwenyu commented 11 months ago

使用最新的代码吧 我已经加了 详细看pytorch的文档 https://github.com/lyuwenyu/RT-DETR/commit/64836918370ef3addf15e88c625300d57828105a

xiaoyangbenjiance commented 10 months ago

你好，我可以看一下你的训练代码吗，我的验证结果和你相差巨大，map0.001不知道哪里出问题了，求助

xiaoyangbenjiance commented 10 months ago

Test: Total time: 0:04:26 (0.4259 s / it) Averaged stats: Accumulating evaluation results... DONE (t=14.13s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.001 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.001 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000

进程已结束，退出代码为 0