search

jihoonerd / Conditional-Motion-In-Betweening

🕹️ Official Implementation of Conditional Motion In-betweening (CMIB) 🏃
https://jihoonerd.github.io/Conditional-Motion-In-Betweening/
123 stars 9 forks source link

Question how is the performance in regards to hand/finger movement and facial expressions? #56

Closed AIMads closed 2 years ago

AIMads commented 2 years ago

I was wondering if the method also works on "finer" detail movement in regards to the smaller body parts as hands and facial expressions.

Cool work ;)

jihoonerd commented 2 years ago

@AIMads Thank you for your interest in this work.

TL;DR

No, the model in this repo does not support that fine details, such as hands or facial expressions.

Limitations

I think the fine-detailed control is more of a data set limitation rather than methods per se. Considering the high quality motion data is mostly retrieved from mocap device, it is still difficult to collect finger or facial muscles. (I think facial muscle data will be far more difficult to get ;) ) However, I think SMPL-X model would be a option if you need to work on detailed motion information. SMPL-X is an extension of SMPL, which only has major joints similar like the kinematic chain in this work. SMPL-H includes finger data on top of the SMPL, and SMPL-X includes facial movements on top of the SMPL-H. But, still I am not sure if there are enough high-quality SMPL-X dataset. As far as I know, the biggest dataset of SMPL-based model is AMASS, but it contains SMPL-H at most.

AIMads commented 2 years ago

Thank you so much for the great answer! Alright very interesting will look into SMPL-X ;)