search

TuSimple / centerformer

Implementation for CenterFormer: Center-based Transformer for 3D Object Detection (ECCV 2022)
MIT License
293 stars 28 forks source link

Evaluation on waymo opendataset #28

Open QingXIA233 opened 1 year ago

QingXIA233 commented 1 year ago

Hello, in order to reproduce the waymo results by my own, I trained CenterFormer on waymo and tried to get the performance evaluation like that shown in the README:

issue

I follwed the instruction from https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/quick_start.md, I already have the gt.bin and preds.bin of CenterFormer, but I ran into this error:

issue1

I wonder whether you encounted this issue before, or maybe I've gone to the wrong way? Really need some help here. Please. Thanks in advance.

DezeZhao commented 1 year ago

hello, have you evaluated waymo with det3d code? can you reproduce the results?

QingXIA233 commented 1 year ago

As I see in the original Det3D repo, https://github.com/poodarchu/Det3D/blob/master/GETTING_STARTED.md. The evaluation is for Nuscenes, lyft and kitti, not for waymo. What my question is: after I train CenterFormer on waymo and get the checkpoint, how could I get the level1 and level2 MAP evaluation results like you have shown in your repo.

DezeZhao commented 1 year ago

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

edwardzhou130 commented 1 year ago

Hello, in order to reproduce the waymo results by my own, I trained CenterFormer on waymo and tried to get the performance evaluation like that shown in the README: issue I follwed the instruction from https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/quick_start.md, I already have the gt.bin and preds.bin of CenterFormer, but I ran into this error: issue1 I wonder whether you encounted this issue before, or maybe I've gone to the wrong way? Really need some help here. Please. Thanks in advance.

It's possible that the Waymo open dataset API wasn't built correctly. You can use their fake pred and gt to test first.

wod
edwardzhou130 commented 1 year ago

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

I haven't previously worked on KITTI. But I believe that due to the significant differences in point cloud style between WOD/nuscenes and KITTI, you will need to make significant changes to the training configs. Can you provide more information about how you've used the code and set the configs?

DezeZhao commented 1 year ago

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

I haven't previously worked on KITTI. But I believe that due to the significant differences in point cloud style between WOD/nuscenes and KITTI, you will need to make significant changes to the training configs. Can you provide more information about how you've used the code and set the configs?

ok thank you for your reply. I can show you the change of code later. canyou help me about another issue about the waymo gt boxes xyzwlhr. why r translate to -r-pi/2. Does this mean translate to camera coordinate? but waht I understand is that it is no need.

zhaowenZhou commented 1 year ago

Hi, did you solve it?

zhaowenZhou commented 1 year ago

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

你好，我搜不到你的微信，请问怎么联系你？