search

Sirui-Xu / InterDiff

[ICCV 2023] Official PyTorch implementation of the paper "InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion"
https://sirui-xu.github.io/InterDiff
MIT License
236 stars 8 forks source link

Evaluate on the GRAB dataset #6

Open mengqing0312 opened 1 year ago

mengqing0312 commented 1 year ago

Thank you for your excellent work and open sourcing it, I was wondering if you will make public the code that was tested on the GRAB dataset?

Sirui-Xu commented 1 year ago

Hello,

We will eventually release everything including the evaluation on the GRAB dataset. However, due to scheduling conflicts, we may not be able to release it this month. Sorry for the delay.

Best

Steve-Tod commented 1 year ago

Hi @Sirui-Xu ,

It seems GRAB data uses SMPL-X instead of SMPL-H used in BEHAVE. Did you transfer them from SMPL-X to SMPL-H using this?

Thank you!

Sirui-Xu commented 1 year ago

Hi @Steve-Tod ,

We do not use this transformation. We copy the body pose directly from the SMPL-H model to the SMPL-X model. However, for the hand and head pose, the approach was to maintain the poses from the last known frame of the previous motion sequence. I will check the details with my collaborator.

This was adopted because the BEHAVE dataset, which was used for training, does not provide detailed hand pose data, as noted in https://github.com/xiexh20/behave-dataset/issues/14. Consequently, the system does not actively generate hand poses. When using the GRAB dataset for evaluation, the primary focus is on the accuracy of the predicted hand joint positions rather than the intricacies of the hand's grasping motions.

Best

Steve-Tod commented 1 year ago

Hi @Sirui-Xu ,

Thanks for the reply! I'll try it out.

Steve-Tod commented 1 year ago

Hi @Sirui-Xu ,

Sorry to interrupt again. I tried re-enacting the SMPL-X parameters from GRAB to SMPL+H models used in InterDiff, however there's a small gap between the mesh vertices from the SMPL-X model and the vertices from SMPL+H model. Like in the following, the one contacting with the object is the mesh vertices from SMPL-X while the one a little bit off is the mesh vertices from SMPL+H.

image

Did you do any special processing to align these two? Thank you in advance!

Sirui-Xu commented 1 year ago

Hi @Steve-Tod ,

That's interesting. We don't have any special programs as I recall. The evaluation on GRAB was done by my co-author, let me check with him. Will get back to you shortly.

Best

Steve-Tod commented 1 year ago

Hi @Steve-Tod ,

That's interesting. We don't have any special programs as I recall. The evaluation on GRAB was done by my co-author, let me check with him. Will get back to you shortly.

Best

Sure, thank you!

Steve-Tod commented 1 year ago

Hi @Sirui-Xu ,

Thanks! I'll give it a try!

z050209 commented 10 months ago

@Steve-Tod were you able to evaluate the GRAB dataset using this pipeline, possible to share the script?

Thanks.