search

tjiiv-cprg / EPro-PnP

[CVPR 2022 Oral, Best Student Paper] EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation
https://www.youtube.com/watch?v=TonBodQ6EUU
Apache License 2.0
1.11k stars 106 forks source link

Can EPro-PnP obtain good detection performance for non-rigid object? Or It is only suitable for rigid object? #39

Open jichaofeng opened 2 years ago

jichaofeng commented 2 years ago

Thanks for sharing your work.

Lakonik commented 2 years ago

In our experiments, non-rigid objects such as pedestrians are involved in the 3D object detection task. You can find the per-category results on the nuScenes leaderboard. The models (EPro-PnP-Det v1 and v2) perform pretty well in these categories, especially in orientation accuracy (mAOE).

Still, I think EPro-PnP can be better viewed as a general loss function. To handle some special objects (e.g., symmetric or non-rigid), you need a proper network architecture to predict the 2D-3D correspondences. As far as I can tell the deformable correspondence network seems a good choice for 3D object detection task.

jichaofeng commented 2 years ago

Thank you very much. We want to ask you a question which is not about EPro-PnP. For a feature pixel, we know its probability distribution of depth. The depth search space is [D1, D2] and we split it in Ns bins, The depth of pixel is the expectation for all depth candidates with probability. if we want to obtain the probability distribution of depth in a full range [0, Dmax] and the range is also split in Ns bins, what should we do? D1 and D2 is less than Dmax.