search

paulchhuang / bstro

The official code for BSTRO in paper: Capturing and Inferring Dense Full-Body Human-Scene Contact, CVPR2022
84 stars 7 forks source link

Getting random output #11

Closed charisoudis closed 9 months ago

charisoudis commented 9 months ago

Hi and thanks for publishing the code of this great work!

I am getting the following weird output from a 3DPW frame: image

Seeing the contact probs, it seems that most are between [0.4, 0.55], so it is like a pseudo-random prediction. I am following your code for loading the model, and preparing the input.

Do you think that this is related to the checkpoint loading of the BSTRO model (I used the metro_3dpw_state_dict.bin)?

Best, Thanos

paulchhuang commented 9 months ago

Hi, If you are following the instructions here: https://github.com/paulchhuang/bstro/blob/main/docs/DEMO.md#human-scene-contact-detection, you should load the hsi_hrnet_3dpw_b32_checkpoint_15.bin downloaded from the RICH website ( hyperlink "BSTRO pre-trained checkpoint"), not metro_3dpw_state_dict.bin. metro_3dpw_state_dict.bin is from another work METRO, which estimates joint positions, not contact. Two models share the same HRNet backbones so we used metro_3dpw_state_dict.bin to initialize its weight during training, but for inference you need only hsi_hrnet_3dpw_b32_checkpoint_15.bin if my memory serves me correctly.

charisoudis commented 9 months ago

That's right, thanks for the response. I got that fixed now!