Open qqfoxmail opened 2 years ago
I also got this problem. And i found that, the loss value did not decrease after 10 epochs.
13%|█▎ | 495/3712 [00:03<00:26, 121.99it/s]epoch: 0 step: 500 and the cls_loss is : 0.8049887084960937 27%|██▋ | 995/3712 [00:07<00:22, 120.07it/s]epoch: 0 step: 1000 and the cls_loss is : 0.13632073974609374 40%|████ | 1489/3712 [00:11<00:17, 124.05it/s]epoch: 0 step: 1500 and the cls_loss is : 0.10595902252197266 54%|█████▍ | 1997/3712 [00:15<00:13, 123.91it/s]epoch: 0 step: 2000 and the cls_loss is : 0.13320989990234375 67%|██████▋ | 2493/3712 [00:19<00:09, 125.33it/s]epoch: 0 step: 2500 and the cls_loss is : 0.13816131591796876 81%|████████ | 2995/3712 [00:24<00:05, 121.45it/s]epoch: 0 step: 3000 and the cls_loss is : 0.11577693939208984 94%|█████████▍| 3492/3712 [00:28<00:01, 126.52it/s]epoch: 0 step: 3500 and the cls_loss is : 0.11067961883544922
Hi,thank you for your work. But some error occurs,I hope you can give me some suggestions. When I train the model.the 3D result is always bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000
I am sorry I did not encounter your problem before. I need more information. I update some codes and readme. You can check them if they can help you organize your project or fix your problem.
I also got this problem. And i found that, the loss value did not decrease after 10 epochs.
13%|█▎ | 495/3712 [00:03<00:26, 121.99it/s]epoch: 0 step: 500 and the cls_loss is : 0.8049887084960937 27%|██▋ | 995/3712 [00:07<00:22, 120.07it/s]epoch: 0 step: 1000 and the cls_loss is : 0.13632073974609374 40%|████ | 1489/3712 [00:11<00:17, 124.05it/s]epoch: 0 step: 1500 and the cls_loss is : 0.10595902252197266 54%|█████▍ | 1997/3712 [00:15<00:13, 123.91it/s]epoch: 0 step: 2000 and the cls_loss is : 0.13320989990234375 67%|██████▋ | 2493/3712 [00:19<00:09, 125.33it/s]epoch: 0 step: 2500 and the cls_loss is : 0.13816131591796876 81%|████████ | 2995/3712 [00:24<00:05, 121.45it/s]epoch: 0 step: 3000 and the cls_loss is : 0.11577693939208984 94%|█████████▍| 3492/3712 [00:28<00:01, 126.52it/s]epoch: 0 step: 3500 and the cls_loss is : 0.11067961883544922
You can check your eval results, Maybe it is good enough. It usually needs only several epochs(maybe 10) to get a good enough results.
I also got this problem. And i found that, the loss value did not decrease after 10 epochs. 13%|█▎ | 495/3712 [00:03<00:26, 121.99it/s]epoch: 0 step: 500 and the cls_loss is : 0.8049887084960937 27%|██▋ | 995/3712 [00:07<00:22, 120.07it/s]epoch: 0 step: 1000 and the cls_loss is : 0.13632073974609374 40%|████ | 1489/3712 [00:11<00:17, 124.05it/s]epoch: 0 step: 1500 and the cls_loss is : 0.10595902252197266 54%|█████▍ | 1997/3712 [00:15<00:13, 123.91it/s]epoch: 0 step: 2000 and the cls_loss is : 0.13320989990234375 67%|██████▋ | 2493/3712 [00:19<00:09, 125.33it/s]epoch: 0 step: 2500 and the cls_loss is : 0.13816131591796876 81%|████████ | 2995/3712 [00:24<00:05, 121.45it/s]epoch: 0 step: 3000 and the cls_loss is : 0.11577693939208984 94%|█████████▍| 3492/3712 [00:28<00:01, 126.52it/s]epoch: 0 step: 3500 and the cls_loss is : 0.11067961883544922
You can check your eval results, Maybe it is good enough. It usually needs only several epochs(maybe 10) to get a good enough results.
Thanks for your reply and update! I have tried with your update version, the problem is solved : ) .
Hi @Laiqingsi and @ShiluYang96,
I am also getting the same problem of zero mAP, although loss in changing throughout the training process. Do you know how can I solve this problem?
2023-07-31 16:26:02,070 INFO Car AP@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000
Thank you so much! Regards, Saket
I printed the result.pt file (49.pt) to check the contents. I got blank output for each sample:
{'name': array([], dtype='<U7'), 'truncated': array([], dtype=float64), 'occluded': array([], dtype=int64), 'alpha': array([], dtype=float32), 'bbox': array([], shape=(0, 4), dtype=float64), 'dimensions': array([], shape=(0, 3), dtype=float32), 'location': array([], shape=(0, 3), dtype=float32), 'rotation_y': array([], dtype=float32), 'score': array([], dtype=float32), 'boxes_lidar': array([], shape=(0, 7), dtype=float32), 'frame_id': 7480}
Does this mean the data is not getting loaded properly or there is some problem in collecting the prediction of the model?
Thank you. Regards, Saket
I printed the result.pt file (49.pt) to check the contents. I got blank output for each sample:
{'name': array([], dtype='<U7'), 'truncated': array([], dtype=float64), 'occluded': array([], dtype=int64), 'alpha': array([], dtype=float32), 'bbox': array([], shape=(0, 4), dtype=float64), 'dimensions': array([], shape=(0, 3), dtype=float32), 'location': array([], shape=(0, 3), dtype=float32), 'rotation_y': array([], dtype=float32), 'score': array([], dtype=float32), 'boxes_lidar': array([], shape=(0, 7), dtype=float32), 'frame_id': 7480}
Does this mean the data is not getting loaded properly or there is some problem in collecting the prediction of the model?
Thank you. Regards, Saket
Hello @schatur2. Did you manage to fix problem with blank output ?
I printed the result.pt file (49.pt) to check the contents. I got blank output for each sample:
{'name': array([], dtype='<U7'), 'truncated': array([], dtype=float64), 'occluded': array([], dtype=int64), 'alpha': array([], dtype=float32), 'bbox': array([], shape=(0, 4), dtype=float64), 'dimensions': array([], shape=(0, 3), dtype=float32), 'location': array([], shape=(0, 3), dtype=float32), 'rotation_y': array([], dtype=float32), 'score': array([], dtype=float32), 'boxes_lidar': array([], shape=(0, 7), dtype=float32), 'frame_id': 7480}
Does this mean the data is not getting loaded properly or there is some problem in collecting the prediction of the model?
Thank you.Regards,Saket
Hello@schatur2. Did you manage to fix problem with blank output ?
I printed the result.pt file (49.pt) to check the contents. I got blank output for each sample:
{'name': array([], dtype='<U7'), 'truncated': array([], dtype=float64), 'occluded': array([], dtype=int64), 'alpha': array([], dtype=float32), 'bbox': array([], shape=(0, 4), dtype=float64), 'dimensions': array([], shape=(0, 3), dtype=float32), 'location': array([], shape=(0, 3), dtype=float32), 'rotation_y': array([], dtype=float32), 'score': array([], dtype=float32), 'boxes_lidar': array([], shape=(0, 7), dtype=float32), 'frame_id': 7480}
Does this mean the data is not getting loaded properly or there is some problem in collecting the prediction of the model?
Thank you. Regards, Saket
Hello, I would like to ask how you obtained these .pt format files? I can only get the results in .pkl format.:)
Hi,thank you for your work. But some error occurs,I hope you can give me some suggestions. When I train the model.the 3D result is always bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.70, 0.70: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000 Car AP_R40@0.70, 0.50, 0.50: bbox AP:0.0000, 0.0000, 0.0000 bev AP:0.0000, 0.0000, 0.0000 3d AP:0.0000, 0.0000, 0.0000