search

open-mmlab / OpenPCDet

OpenPCDet Toolbox for LiDAR-based 3D Object Detection.
Apache License 2.0
4.62k stars 1.29k forks source link

Low accuracy when training on waymo dataset #370

Closed LZDSJTU closed 2 years ago

LZDSJTU commented 3 years ago

Hi, I add the recent update on the waymo dataset to the previous version of OpenPCDet (maybe June2020 version), and train second/PartA2 on the waymo dataset. I do not modify the preprocess script and model cfg for waymo.

However, the accuracy is much lower. For second, the final accuracy is as follows: 742 2020-11-24 08:00:32,763 INFO ****Evaluation done.** 743 2020-11-24 09:47:11,516 INFO EPOCH 0 EVALUATION ** 744 2020-11-24 09:48:05,627 INFO Performance of EPOCH 0 ***** 745 2020-11-24 09:48:05,627 INFO Generate label finished(sec_per_example: 0.0068 second). 746 2020-11-24 09:48:05,628 INFO recall_roi_0.3: 0.000000 (0 / 296561) 747 2020-11-24 09:48:05,628 INFO recall_rcnn_0.3: 0.689605 (204510 / 296561) 748 2020-11-24 09:48:05,628 INFO recall_roi_0.5: 0.000000 (0 / 296561) 749 2020-11-24 09:48:05,628 INFO recall_rcnn_0.5: 0.574320 (170321 / 296561) 750 2020-11-24 09:48:05,628 INFO recall_roi_0.7: 0.000000 (0 / 296561) 751 2020-11-24 09:48:05,628 INFO recall_rcnn_0.7: 0.316451 (93847 / 296561) 752 2020-11-24 09:48:05,637 INFO Average predicted number of objects(7998 samples): 100.557 753 2020-11-24 09:51:25,703 INFO 754 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/AP: 0.3911 755 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/APH: 0.3855 756 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/AP: 0.3620 757 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/APH: 0.3568 758 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/AP: 0.3631 759 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/APH: 0.2795 760 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/AP: 0.3308 761 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/APH: 0.2545 762 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/AP: 0.0000 763 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/APH: 0.0000 764 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/AP: 0.0000 765 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/APH: 0.0000 766 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/AP: 0.2761 767 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/APH: 0.2636 768 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/AP: 0.2757 769 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/APH: 0.2632

I'd like to ask what may cause this problem.

Thank you very much.

r-sy commented 2 years ago

@r-sy Hi, I also use waymo-open-dataset-tf-2-5-0 and the numpy version is 1.19.5. But I do not run code nomally and generate this error:"ImportError: numpy.core.multiarray failed to import". Can u tell me your numpy version or other suggestions? Thank u very much!!!

You can try to install SharedArray==3.1.0.

jwma0725 commented 2 years ago

Thanks!!! It works well!

------------------ 原始邮件 ------------------ 发件人: "open-mmlab/OpenPCDet" @.>; 发送时间: 2022年5月6日(星期五) 凌晨0:19 @.>; @.**@.>; 主题: Re: [open-mmlab/OpenPCDet] Low accuracy when training on waymo dataset (#370)

@r-sy Hi, I also use waymo-open-dataset-tf-2-5-0 and the numpy version is 1.19.5. But I do not run code nomally and generate this error:"ImportError: numpy.core.multiarray failed to import". Can u tell me your numpy version or other suggestions? Thank u very much!!!

You can try to install SharedArray==3.1.0.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

carrolltm commented 1 year ago

Hi, I also have this problem when I use waymo v1.0 for object detection(Start the waymo evaluation... Number:(pd, 1322657)VS. (gt,0))，i also use the environment，Can you tell me how you finally solved it?Thank u very much!!! image

RookieXwc commented 1 year ago

@r-sy Hi, I also use waymo-open-dataset-tf-2-5-0 and the numpy version is 1.19.5. But I do not run code nomally and generate this error:"ImportError: numpy.core.multiarray failed to import". Can u tell me your numpy version or other suggestions? Thank u very much!!!

You can try to install SharedArray==3.1.0.

I have the same issue with jwma0725, and install SharedArray==3.1.0 work!!!!!!!