search

yixchen / holistic_scene_human

MIT License
48 stars 2 forks source link

How is the HOI prior obtained? #4

Closed xiexh20 closed 2 years ago

xiexh20 commented 2 years ago

Dear authors,

Thanks for the great work! I am impressed by the results you obtained. But I have a question about the interaction term used in your optimization pipeline. In the main paper after equation 5, you said the likelihood can be learned from 3D HOI datasets, can you specify which dataset you used to learn this prior? I also checked the source code here, how are these mean and variances computed?

Thank you very much for your help!

Best,

yixchen commented 2 years ago

HI,

The dataset we used to capture the human-object interaction is collected from a video game Grand Theft Auto V, similar to GTA Indoor Motion dataset (GTA-IM) by Cao et al. Our dataset is under review. Still, you can get similar data from the existing dataset. The mean and variance are computed with 3D human skeletons and object bounding boxes by fitting a Gaussian mixture model between the human and object key points. For example, the sitting relation is calculated between the hip joint and the chair bounding box center. Such probabilistic models can be modified according to which granularity you want to model the HOI relationship.

Hope this helps.

xiexh20 commented 2 years ago

Thank you very much for your detailed response. Now I have a better understanding of the method.

Best of luck with your dataset review!