search

Nightmare-n / GD-MAE

GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds (CVPR 2023)
Apache License 2.0
114 stars 6 forks source link

evaluate AP is all 0 #2

Closed Jung-jongwon closed 1 year ago

Jung-jongwon commented 1 year ago

Hi, thank you for your work. I got all AP is 0 when i train graph_rcnn_voi. Can you tell me what is the problem??

Car AP@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000

Nightmare-n commented 1 year ago

Please use the command bash scripts/dist_ts_train.sh to train the graph_rcnn_vo/voi separately. You can directly load the weights of the one-stage model (second_mini+centernet) and then only train the refinement network.

SiHengHeHSH commented 1 year ago

I also got all AP is 0 when i train graph_rcnn_voi.

Nightmare-n commented 1 year ago

hi, how do you train the model? Can you also provide the scripts?

SiHengHeHSH commented 1 year ago

I just debug in my computer,and I don't use scripts.A train in the mini-kitti show that the AP is all 0.

SiHengHeHSH commented 1 year ago

just like him

2023-05-16 16:31:24,218 INFO **Start evaluation kitti_models/graph_rcnn_po(default)** 2023-05-16 16:31:24,219 INFO Loading KITTI dataset 2023-05-16 16:31:24,220 INFO Total samples for KITTI dataset: 43 2023-05-16 16:31:24,222 INFO ==> Loading parameters from checkpoint /home/hsh/下载/GD-MAE/output/kitti_models/graph_rcnn_po/default/ckpt/checkpoint_epoch_1.pth to GPU 2023-05-16 16:31:24,351 INFO ==> Checkpoint trained from version: pcdet+0.5.1+abd05ce 2023-05-16 16:31:24,380 INFO ==> Done (loaded 301/301) 2023-05-16 16:31:24,391 INFO * EPOCH 1 EVALUATION *** eval: 100%|██████████| 43/43 [00:07<00:00, 5.50it/s, recall_0.3=(43, 36) / 128] 2023-05-16 16:31:32,210 INFO * Performance of EPOCH 1 *** 2023-05-16 16:31:32,210 INFO Run time per sample: 0.1535 second. 2023-05-16 16:31:32,210 INFO Generate label finished(sec_per_example: 0.1817 second). 2023-05-16 16:31:32,210 INFO recall_roi_0.3: 0.335938 2023-05-16 16:31:32,210 INFO recall_rcnn_0.3: 0.281250 2023-05-16 16:31:32,210 INFO recall_roi_0.5: 0.039062 2023-05-16 16:31:32,211 INFO recall_rcnn_0.5: 0.023438 2023-05-16 16:31:32,211 INFO recall_roi_0.7: 0.000000 2023-05-16 16:31:32,211 INFO recall_rcnn_0.7: 0.000000 2023-05-16 16:31:32,211 INFO Average predicted number of objects(43 samples): 0.000 2023-05-16 16:31:52,293 INFO Car AP@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000

2023-05-16 16:31:52,293 INFO Result is save to /home/hsh/下载/GD-MAE/output/kitti_models/graph_rcnn_po/default/eval/eval_with_train/epoch_1/val 2023-05-16 16:31:52,293 INFO ****Evaluation done.***** 2023-05-16 16:31:52,298 INFO Epoch 1 has been evaluated 2023-05-16 16:33:22,376 INFO **End evaluation kitti_models/graph_rcnn_po(default)**

Nightmare-n commented 1 year ago

Hi, I am not sure whether the model could achieve desirable results with only 43 training samples and 1 epoch. Maybe you could try it with more samples and epochs.

SiHengHeHSH commented 1 year ago

2023-05-18 15:13:53,805 INFO * Performance of EPOCH 1 *** 2023-05-18 15:13:53,806 INFO Run time per sample: 0.1001 second. 2023-05-18 15:13:53,806 INFO Generate label finished(sec_per_example: 0.1035 second). 2023-05-18 15:13:53,806 INFO recall_roi_0.3: 0.000000 2023-05-18 15:13:53,806 INFO recall_rcnn_0.3: 0.000000 2023-05-18 15:13:53,806 INFO recall_roi_0.5: 0.000000 2023-05-18 15:13:53,806 INFO recall_rcnn_0.5: 0.000000 2023-05-18 15:13:53,806 INFO recall_roi_0.7: 0.000000 2023-05-18 15:13:53,806 INFO recall_rcnn_0.7: 0.000000 2023-05-18 15:13:53,816 INFO Average predicted number of objects(3769 samples): 0.000 2023-05-18 15:14:06,675 INFO Car AP@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000

2023-05-18 15:14:06,683 INFO Result is save to /share/home/105324/GD-MAE/output/kitti_models/graph_rcnn_voi/default/eval/eval_with_train/epoch_1/val 2023-05-18 15:14:06,683 INFO ****Evaluation done.***** 2023-05-18 15:14:06,715 INFO Epoch 1 has been evaluated 2023-05-18 15:15:36,810 INFO **End evaluation kitti_models/graph_rcnn_voi(default)**

SiHengHeHSH commented 1 year ago

Maybe even if I train 80 epoch，the sequence could be same.But I don't know why?

Nightmare-n commented 1 year ago

Hi, please refer to issue#12, which may be helpful.