search

Arthur151 / ROMP

Monocular, One-stage, Regression of Multiple 3D People and their 3D positions & trajectories in camera & global coordinates. ROMP[ICCV21], BEV[CVPR22], TRACE[CVPR2023]
https://www.yusun.work/
Apache License 2.0
1.36k stars 231 forks source link

H36M SMPL parameters and results.npz file analysis - Request #421

Closed Dipankar1997161 closed 1 year ago

Dipankar1997161 commented 1 year ago

Hello @Arthur151, Thank you for the work you posted here.

I went through the following page https://github.com/Arthur151/ROMP/blob/12a624f00be92d1f7195f24106d151d2f26961f4/docs/config_guide.md, however, there are few arrays generated which are not mentioned in it. Could you clarify those to me?

  1. "global-orient"

  2. trans (3,) # rough 3D translation converted from the estimated camera parameters. - in the npz_file 'cam-trans' which is not an issue. My question for this is: " from the estimated camera parameters" <<-- from where these estemated camera parameters were defined for ROMP demo.

  3. Lastly, I am working on a project based on h36m for view-synthesis, and for preparation purposes, I require 4 major metadata : smpl-betas, and poses (got these 2 from the npz file after running the demo on 1 h36m scenario video).

However, I also require the intrinsic and extrinsic camera parameters for the same (I ran the demo romp on h36m subject 11 walking scenario). Is there a way to generate these for h36m dataset, lets say for the subject 11 walking scenario? Here is an example of what I require for the synthesis.

metadata ss

I would love to hear from you on this matter. THANK YOU ONCE AGAIN

Arthur151 commented 1 year ago

@Dipankar1997161 Sure, here is the answers:

  1. the 'global-orient' is the first 3 parameters of 72-dim SMPL pose parameters.
  2. We employ a pre-defined fixed camera model, and all predictions are in this camere model space. The camera is assumed to be of FOV = 60 degree, and the focal length is calculated via https://github.com/Arthur151/ROMP/blob/12a624f00be92d1f7195f24106d151d2f26961f4/romp/lib/config.py#L58 The extrinsic is simple, always np.eye(4).
shalseban commented 1 year ago

I have struggling with a similar issue. I might have missed the answer to this - the extrinsic matrix is np.eye(4), whats the intrinsic matrix?

Dipankar1997161 commented 1 year ago

I have struggling with a similar issue. I might have missed the answer to this - the extrinsic matrix is np.eye(4), whats the intrinsic matrix?

The intrinsic is basically your inage center as cx and Cy Next thr focal length can be taken from the config.py as mentioned in the above comment. This was you can setup the intrisic matrix.

Otherwise use the values ROMP have given in the output file and convert it from weak perspective camera to a regular exrinsic. That's the other way