search

Arthur151 / ROMP

Monocular, One-stage, Regression of Multiple 3D People and their 3D positions & trajectories in camera & global coordinates. ROMP[ICCV21], BEV[CVPR22], TRACE[CVPR2023]
https://www.yusun.work/
Apache License 2.0
1.33k stars 229 forks source link

question about the difference between the intrinsics and extrinsics calibrated and estimated by ROMP #324

Open sycamore27 opened 2 years ago

sycamore27 commented 2 years ago

Hello, I use ROMP to estimate the SMPL and use it to render novel view pictures. But I found that if using the intrinsics and extrinsics estimated by ROMP can correctly generate the bounding box in 2D picture, but using intrinsics and extrinsics calibrated by the camera ourselves will be wrong, which is really inexplicable. Why can't I use the intrinsics and extrinsics calibrated by the camera? Do you know the possible reason? Thank you very much!

Arthur151 commented 2 years ago

Hi. You can use our estimated 2D keypoints and 3D keypoints to re-calcualte the 3D translation of people via PnP. The reason of failure is that the 3D translation we estimated is just for our pre-defined camera, instead of any other pecific camera. It would be helpful to determine the bug is you can provide the details of your intrinsics and extrinsics. The PnP could refer to https://github.com/Arthur151/ROMP/blob/bafc86897c387caae125e7119b31dc30ee317bf0/simple_romp/romp/utils.py#L331 You can replace the camMat with your intrinsics

sycamore27 commented 2 years ago

Thank you for your reply! The correct 2D bounding box can be obtained by recalculating 3D translation with my intrinsics by the _estimate_translationcv2 function.

However, there is another question. How can I get the correct smpl and corresponding 2D bounding box when I have set an exact camera position, in other words, given an exact extrinsics? Thank you very much!

Arthur151 commented 2 years ago

Sorry, this feature is not supported by the current version, but will be supported by the next version. Best