Evaluate on the GRAB dataset

[mengqing0312]

Thank you for your excellent work and open sourcing it, I was wondering if you will make public the code that was tested on the GRAB dataset?

[Sirui-Xu]

Hello,

We will eventually release everything including the evaluation on the GRAB dataset. However, due to scheduling conflicts, we may not be able to release it this month. Sorry for the delay.

Best

[Steve-Tod]

Hi @Sirui-Xu ,

It seems GRAB data uses SMPL-X instead of SMPL-H used in BEHAVE. Did you transfer them from SMPL-X to SMPL-H using this?

Thank you!

[Sirui-Xu]

Hi @Steve-Tod ,

We do not use this transformation. We copy the body pose directly from the SMPL-H model to the SMPL-X model. However, for the hand and head pose, the approach was to maintain the poses from the last known frame of the previous motion sequence. I will check the details with my collaborator.

This was adopted because the BEHAVE dataset, which was used for training, does not provide detailed hand pose data, as noted in https://github.com/xiexh20/behave-dataset/issues/14. Consequently, the system does not actively generate hand poses. When using the GRAB dataset for evaluation, the primary focus is on the accuracy of the predicted hand joint positions rather than the intricacies of the hand's grasping motions.

Best

[Steve-Tod]

Hi @Sirui-Xu ,

Thanks for the reply! I'll try it out.

[Steve-Tod]

Hi @Sirui-Xu ,

Sorry to interrupt again. I tried re-enacting the SMPL-X parameters from GRAB to SMPL+H models used in InterDiff, however there's a small gap between the mesh vertices from the SMPL-X model and the vertices from SMPL+H model. Like in the following, the one contacting with the object is the mesh vertices from SMPL-X while the one a little bit off is the mesh vertices from SMPL+H.

Did you do any special processing to align these two? Thank you in advance!

[Sirui-Xu]

Hi @Steve-Tod ,

That's interesting. We don't have any special programs as I recall. The evaluation on GRAB was done by my co-author, let me check with him. Will get back to you shortly.

Best

[Steve-Tod]

Hi @Steve-Tod ,

That's interesting. We don't have any special programs as I recall. The evaluation on GRAB was done by my co-author, let me check with him. Will get back to you shortly.

Best

Sure, thank you!

[Steve-Tod]

Hi @Sirui-Xu ,

Thanks! I'll give it a try!

[z050209]

@Steve-Tod were you able to evaluate the GRAB dataset using this pipeline, possible to share the script?

Thanks.