search

scaleapi / pandaset-devkit

Other
250 stars 61 forks source link

How to evaluating results? #120

Open Leozyc-waseda opened 2 years ago

Leozyc-waseda commented 2 years ago

Hi,there I am trying to test kitti trained models like pv_rcnn on pandaset. it looks like there is no standard evaluation metric yet.

can you tell me how to do it？

Joeless commented 2 years ago

Hi,there I am trying to test kitti trained models like pv_rcnn on pandaset. it looks like there is no standard evaluation metric yet.

can you tell me how to do it？

I also have this question,how you solve it?

Leozyc-waseda commented 2 years ago

According to the pandaset paper, they are using kitti evaluation metric.

But I don't know how to convert to Kitti format, and the paper doesn't give a tutorial, I'm still stuck in this place, waiting for the official evaluation metric.

By the way, after I converted Pandaset's data into Bin format and tested, there is no result...

Joeless commented 2 years ago

Do you have wechat?, I am stuck int this place too I want to talk with you~ my wechat: MegaloAiming

Joeless commented 2 years ago

According to the pandaset paper, they are using kitti evaluation metric.

But I don't know how to convert to Kitti format, and the paper doesn't give a tutorial, I'm still stuck in this place, waiting for the official evaluation metric.

By the way, after I converted Pandaset's data into Bin format and tested, there is no result...

Do you get the AP? I reappear pandaset on 3D model ,but I get the higher AP in car ,the lower AP in Ped&cyclist

Leozyc-waseda commented 2 years ago

@Joeless Sorry for the late reply, you can contact me by email. I tried converting pandas to kitti's bin format, but I didn't get any results (AP=0), and then I gave up Debug. BTW, can you share your approach?

Joeless commented 2 years ago

@Joeless Sorry for the late reply, you can contact me by email. I tried converting pandas to kitti's bin format, but I didn't get any results (AP=0), and then I gave up Debug. BTW, can you share your approach?

I just transfrom to kitti's format,using kitti eval

Leozyc-waseda commented 2 years ago

Can you share your code? :)