search

yz-cnsdqz / PSI-release

official implementation of CVPR'20 oral paper: Generating 3D People in Scenes without People.: https://ps.is.tuebingen.mpg.de/publications/smpl-x-conditional-vae-prox-scene-constraints
Other
149 stars 16 forks source link

How to fine corresponding frame for each training sample? #8

Closed sunshineatnoon closed 4 years ago

sunshineatnoon commented 4 years ago

Hi! Thanks for open-sourcing this awesome project. After exploring the PROXE_ExpansionPack, I found that each training sample includes depth, semantics, is it possible to know the corresponding frame name for each training sample? Or does every frame in the PROX dataset is included in the PROXE_ExpansionPack? Thanks.

yz-cnsdqz commented 4 years ago

Thanks for your interests.

All frames are randomly shuffled, and there is no info of which frame it comes from. I think you could use the provided code to process PROX again.

sunshineatnoon commented 4 years ago

Hi, Thanks so much for your quick reply. I took a look at the processing file, it seems the BatchGeneratorWithSceneMeshMatfile class here needs some reconstruction mat files, could you please tell me where I can find them? I didn't see them in the PROX dataset release. I think the snapshot_for_testing folder includes some mat files, but only for the testing scenes.

One thing I am particular interested is the transformation used to project fitted SMPL models to the world coordinates, I know the quantitative PROX dataset includes such transformation (the vicon2scene.json file), but couldn't find any for the qualitative PROX dataset, I would appreciate if you could tell where I can get the transformations. Thanks!

yz-cnsdqz commented 4 years ago

You can download the prox qualitative data first, and generate mat files by yourself, using utils_prox_snapshots_*.py. There you could find what you would like to get, e.g. transforming smplx parameters into different camera coordinates.

sunshineatnoon commented 4 years ago

Thanks! I am able to align the SMPL model with the scene now