Waymo Results: mAP for all classes or only for the vehicle class

[abhi1kumar]

Hi PCT authors, I had a small query regarding the Waymo results. Table 7 of your paper reports the mAP on Waymo dataset. Do you report the mAP/ mAPH of all the classes or is it only the mAP/mAPH for the vehicle (car) class ?

PS- Another paper CaDDN only reports mAP on the vehicle (car) class in their Table 2.

[Willy0919]

As we take coordinate-based method as our baseline, we follow the common training and evaluation strategy which only takes the vehichle patches as inputs and reports results on vehicle class. By the way, we did not adjust the training parameters on Waymo dataset and use the same configuration as KITTI. You can also adjust it for a more suitable configuration on Waymo dataset.

[abhi1kumar]

@Willy0919 Thankyou for answering my query.

As we take coordinate-based method as our baseline, we follow the common training and evaluation strategy which only takes the vehichle patches as inputs and reports results on vehicle class.

It makes a lot of sense to carry out the evaluation on the vehicle class.

By the way, we did not adjust the training parameters on Waymo dataset and use the same configuration as KITTI. You can also adjust it for a more suitable configuration on Waymo dataset.

Sounds good.

One related final question: Table 7 of your paper reports mAP on the single vehicle-class. Was your Waymo model trained on single-class or for multi-class ? In case you use multiple classes for training, what classes did you use for training this Waymo model. Here are some of the options I could think of

• All classes in Waymo - Vehicle, Pedestrian, Cyclist as well as Sign (i.e., four foreground classes)
• Only KITTI Classes - Vehicle, Pedestrian, and Cyclist (i.e., three foreground classes)

[Willy0919]

Hi,

We only train single vehicle-class on Waymo dataset. As reported in CaDDN, If you want to train multi-class on Waymo dataset, I suggest you only train KITTI classes whose convertion can be seen in convertor.py.

-- Best regards, Li Wang

-----原始邮件----- 发件人:"Abhinav Kumar (अभिनव कुमार)" @.> 发送时间:2022-02-07 11:15:11 (星期一) 收件人: amazon-research/progressive-coordinate-transforms @.> 抄送: "Li Wang" @.>, Mention @.> 主题: Re: [amazon-research/progressive-coordinate-transforms] Waymo Results: mAP for all classes or only for the vehicle class (Issue #17)

@Willy0919 Thankyou for answering my query.

As we take coordinate-based method as our baseline, we follow the common training and evaluation strategy which only takes the vehichle patches as inputs and reports results on vehicle class.

It makes a lot of sense to carry out the evaluation on the vehicle class.

By the way, we did not adjust the training parameters on Waymo dataset and use the same configuration as KITTI. You can also adjust it for a more suitable configuration on Waymo dataset.

Sounds good.

One related final question: Table 7 of your paper reports mAP on the single vehicle-class. Was your Waymo model trained on single-class or for multi-class ? In case you use multiple classes for training, what classes did you use for training this Waymo model. Here are some of the options I could think of

All classes in Waymo - Vehicle, Pedestrian, Cyclist as well as Sign (i.e., four foreground classes) Only KITTI Classes - Vehicle, Pedestrian, and Cyclist (i.e., three foreground classes)

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>

[abhi1kumar]

We only train single vehicle-class on Waymo dataset. As reported in CaDDN, If you want to train multi-class on Waymo dataset, I suggest you only train KITTI classes whose convertion can be seen in convertor.py.

@Willy0919 Thankyou for letting this know. I deeply appreciate it.