search

zhanggang001 / HEDNet

HEDNet (NeurIPS 2023) & SAFDNet (CVPR 2024 Oral)
Apache License 2.0
88 stars 7 forks source link

我使用nuscenes-mini训练了20个epoch，但是得到了较低的map #17

Open Hichengdong opened 3 weeks ago

Hichengdong commented 3 weeks ago

这是因为mini数据集原因吗？有没有训练好的模型提供呢？ Filtering predictions => Original number of boxes: 12177 => After distance based filtering: 9922 => After LIDAR points based filtering: 9922 => After bike rack filtering: 9915 Filtering ground truth annotations => Original number of boxes: 4441 => After distance based filtering: 3785 => After LIDAR points based filtering: 3393 => After bike rack filtering: 3393 Accumulating metric data... Calculating metrics... Saving metrics to: /home/cd/mmdetection3d/HEDNet/output/eval/eval_with_train/epoch_20/val/final_result/data mAP: 0.1458 mATE: 0.6777 mASE: 0.5944 mAOE: 1.0828 mAVE: 0.9138 mAAE: 0.5996 NDS: 0.1944 Eval time: 1.2s

Per-class results: Object Class AP ATE ASE AOE AVE AAE car 0.537 0.379 0.207 1.283 0.512 0.158 truck 0.156 0.488 0.310 1.114 0.136 0.015 bus 0.190 1.152 0.193 0.928 2.461 0.789 trailer 0.000 1.000 1.000 1.000 1.000 1.000 construction_vehicle 0.000 1.000 1.000 1.000 1.000 1.000 pedestrian 0.571 0.177 0.299 0.757 1.120 0.747 motorcycle 0.005 0.379 0.421 1.663 0.081 0.088 bicycle 0.000 1.000 1.000 1.000 1.000 1.000 traffic_cone 0.000 0.202 0.513 nan nan nan barrier 0.000 1.000 1.000 1.000 nan nan 2024-06-04 13:27:53,761 INFO ----------------Nuscene detection_cvpr_2019 results----------------- car error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.38, 0.21, 1.28, 0.51, 0.16 | 31.15, 50.00, 64.40, 69.14 | mean AP: 0.536750495327818 truck error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.49, 0.31, 1.11, 0.14, 0.01 | 3.91, 16.86, 20.77, 20.77 | mean AP: 0.15576120980287722 construction_vehicle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 bus error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.15, 0.19, 0.93, 2.46, 0.79 | 0.00, 2.23, 36.83, 36.83 | mean AP: 0.1897158684027326 trailer error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 barrier error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, nan, nan | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 motorcycle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.38, 0.42, 1.66, 0.08, 0.09 | 0.10, 0.32, 0.39, 1.18 | mean AP: 0.004950332753012125 bicycle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 pedestrian error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.18, 0.30, 0.76, 1.12, 0.75 | 54.40, 55.48, 56.74, 61.79 | mean AP: 0.5710285108655466 traffic_cone error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.20, 0.51, nan, nan, nan | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 --------------average performance------------- trans_err: 0.6777 scale_err: 0.5944 orient_err: 1.0828 vel_err: 0.9138 attr_err: 0.5996 mAP: 0.1458 NDS: 0.1944

2024-06-04 13:27:53,762 INFO Result is saved to /home/cd/mmdetection3d/HEDNet/output/eval/eval_with_train/epoch_20/val 2024-06-04 13:27:53,762 INFO ****Evaluation done.***** 2024-06-04 13:27:53,764 INFO Epoch 20 has been evaluated 2024-06-04 13:28:23,795 INFO **End evaluation cfgs/safdnet_models/safdnet_20e_nuscenes(default)**

zhanggang001 commented 1 week ago

这是因为mini数据集原因吗？有没有训练好的模型提供呢？ Filtering predictions => Original number of boxes: 12177 => After distance based filtering: 9922 => After LIDAR points based filtering: 9922 => After bike rack filtering: 9915 Filtering ground truth annotations => Original number of boxes: 4441 => After distance based filtering: 3785 => After LIDAR points based filtering: 3393 => After bike rack filtering: 3393 Accumulating metric data... Calculating metrics... Saving metrics to: /home/cd/mmdetection3d/HEDNet/output/eval/eval_with_train/epoch_20/val/final_result/data mAP: 0.1458 mATE: 0.6777 mASE: 0.5944 mAOE: 1.0828 mAVE: 0.9138 mAAE: 0.5996 NDS: 0.1944 Eval time: 1.2s

Per-class results: Object Class AP ATE ASE AOE AVE AAE car 0.537 0.379 0.207 1.283 0.512 0.158 truck 0.156 0.488 0.310 1.114 0.136 0.015 bus 0.190 1.152 0.193 0.928 2.461 0.789 trailer 0.000 1.000 1.000 1.000 1.000 1.000 construction_vehicle 0.000 1.000 1.000 1.000 1.000 1.000 pedestrian 0.571 0.177 0.299 0.757 1.120 0.747 motorcycle 0.005 0.379 0.421 1.663 0.081 0.088 bicycle 0.000 1.000 1.000 1.000 1.000 1.000 traffic_cone 0.000 0.202 0.513 nan nan nan barrier 0.000 1.000 1.000 1.000 nan nan 2024-06-04 13:27:53,761 INFO ----------------Nuscene detection_cvpr_2019 results----------------- car error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.38, 0.21, 1.28, 0.51, 0.16 | 31.15, 50.00, 64.40, 69.14 | mean AP: 0.536750495327818 truck error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.49, 0.31, 1.11, 0.14, 0.01 | 3.91, 16.86, 20.77, 20.77 | mean AP: 0.15576120980287722 construction_vehicle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 bus error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.15, 0.19, 0.93, 2.46, 0.79 | 0.00, 2.23, 36.83, 36.83 | mean AP: 0.1897158684027326 trailer error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 barrier error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, nan, nan | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 motorcycle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.38, 0.42, 1.66, 0.08, 0.09 | 0.10, 0.32, 0.39, 1.18 | mean AP: 0.004950332753012125 bicycle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 pedestrian error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.18, 0.30, 0.76, 1.12, 0.75 | 54.40, 55.48, 56.74, 61.79 | mean AP: 0.5710285108655466 traffic_cone error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.20, 0.51, nan, nan, nan | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 --------------average performance------------- trans_err: 0.6777 scale_err: 0.5944 orient_err: 1.0828 vel_err: 0.9138 attr_err: 0.5996 mAP: 0.1458 NDS: 0.1944

2024-06-04 13:27:53,762 INFO Result is saved to /home/cd/mmdetection3d/HEDNet/output/eval/eval_with_train/epoch_20/val 2024-06-04 13:27:53,762 INFO **Evaluation done.* 2024-06-04 13:27:53,764 INFO Epoch 20 has been evaluated 2024-06-04 13:28:23,795 INFO End evaluation cfgs/safdnet_models/safdnet_20e_nuscenes(default)**

Why did you use the mini-dataset? We did not try that. The model will be released asap. We will let you know after we released the checkpoint.

Hichengdong commented 1 week ago

这是因为mini数据集原因吗？有没有训练好的模型提供呢？ Filtering predictions => Original number of boxes: 12177 => After distance based filtering: 9922 => After LIDAR points based filtering: 9922 => After bike rack filtering: 9915 Filtering ground truth annotations => Original number of boxes: 4441 => After distance based filtering: 3785 => After LIDAR points based filtering: 3393 => After bike rack filtering: 3393 Accumulating metric data... Calculating metrics... Saving metrics to: /home/cd/mmdetection3d/HEDNet/output/eval/eval_with_train/epoch_20/val/final_result/data mAP: 0.1458 mATE: 0.6777 mASE: 0.5944 mAOE: 1.0828 mAVE: 0.9138 mAAE: 0.5996 NDS: 0.1944 Eval time: 1.2s Per-class results: Object Class AP ATE ASE AOE AVE AAE car 0.537 0.379 0.207 1.283 0.512 0.158 truck 0.156 0.488 0.310 1.114 0.136 0.015 bus 0.190 1.152 0.193 0.928 2.461 0.789 trailer 0.000 1.000 1.000 1.000 1.000 1.000 construction_vehicle 0.000 1.000 1.000 1.000 1.000 1.000 pedestrian 0.571 0.177 0.299 0.757 1.120 0.747 motorcycle 0.005 0.379 0.421 1.663 0.081 0.088 bicycle 0.000 1.000 1.000 1.000 1.000 1.000 traffic_cone 0.000 0.202 0.513 nan nan nan barrier 0.000 1.000 1.000 1.000 nan nan 2024-06-04 13:27:53,761 INFO ----------------Nuscene detection_cvpr_2019 results----------------- car error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.38, 0.21, 1.28, 0.51, 0.16 | 31.15, 50.00, 64.40, 69.14 | mean AP: 0.536750495327818 truck error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.49, 0.31, 1.11, 0.14, 0.01 | 3.91, 16.86, 20.77, 20.77 | mean AP: 0.15576120980287722 construction_vehicle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 bus error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.15, 0.19, 0.93, 2.46, 0.79 | 0.00, 2.23, 36.83, 36.83 | mean AP: 0.1897158684027326 trailer error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 barrier error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, nan, nan | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 motorcycle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.38, 0.42, 1.66, 0.08, 0.09 | 0.10, 0.32, 0.39, 1.18 | mean AP: 0.004950332753012125 bicycle error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 1.00, 1.00, 1.00, 1.00, 1.00 | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 pedestrian error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.18, 0.30, 0.76, 1.12, 0.75 | 54.40, 55.48, 56.74, 61.79 | mean AP: 0.5710285108655466 traffic_cone error@trans, scale, orient, vel, attr | AP@0.5, 1.0, 2.0, 4.0 0.20, 0.51, nan, nan, nan | 0.00, 0.00, 0.00, 0.00 | mean AP: 0.0 --------------average performance------------- trans_err: 0.6777 scale_err: 0.5944 orient_err: 1.0828 vel_err: 0.9138 attr_err: 0.5996 mAP: 0.1458 NDS: 0.1944 2024-06-04 13:27:53,762 INFO Result is saved to /home/cd/mmdetection3d/HEDNet/output/eval/eval_with_train/epoch_20/val 2024-06-04 13:27:53,762 INFO **Evaluation done.* 2024-06-04 13:27:53,764 INFO Epoch 20 has been evaluated 2024-06-04 13:28:23,795 INFO End evaluation cfgs/safdnet_models/safdnet_20e_nuscenes(default)**

Why did you use the mini-dataset? We did not try that. The model will be released asap. We will let you know after we released the checkpoint.

好的，谢谢你，我仅有一张NVIDIA 4060ti GPU，训练完整数据集的时间花费太多了。

ouyangziyao commented 3 days ago

老哥，跑过完整的吗？我也没有卡。看该论文都是8卡，bs为16，感觉模型很大

Hichengdong commented 2 days ago

老哥，跑过完整的吗？我也没有卡。看该论文都是8卡，bs为16，感觉模型很大

没有呢，我打算自己想点简单的点子做一下，没卡的话，这两三年来新出的算法基本上是跑不了。