Shimingyi / MotioNet

A deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video [ToG 2020]
https://rubbly.cn/publications/motioNet/
BSD 2-Clause "Simplified" License
554 stars 82 forks source link

How to obtain smooth animation? #42

Closed lakpa-tamang9 closed 2 years ago

lakpa-tamang9 commented 2 years ago

The results for the wild are very different than presented in the paper. Open pose base key points extraction was fed as input to the model along with the confidence values as explained, however, the results are horrific.

Shimingyi commented 2 years ago

Hi @lakpa-tamang9 , there are potential problems in the current version to make it hard to be robust on all wild video, like different/moving camera viewpoints, noticeable data noise, and different 2d detection distribution on the original image, etc. These variables will have more effects on our 2d-position-based model compared to the image-feature-based model, it's not good on real applications though our model is lighter. Thus, I am developing an advanced version of MotioNet to obtain more stable performance, hope it can help you more.

lucasjinreal commented 2 years ago

@Shimingyi Hi. will your more advanced version MotioNet open source as well? Really hope new motionet can set a new standard in this field. Currently most open sourced version are not very practical in many aspect.

Shimingyi commented 2 years ago

@jinfagang Yes, of course ; ) Stable and reliable 3d pose estimation is challenging, but every year/month, many new ideas inspire us, and we learn a lot from their open-sourced code. Thankful for that, we will also open-source new version and contribute to the community.

lucasjinreal commented 2 years ago

@Shimingyi thanks. Do u have a roughly timeline for this?

anas-zafar commented 1 year ago

@Shimingyi any updates on the advanced version MotioNet? Thanks