FORTH-ModelBasedTracker / MocapNET

We present MocapNET, a real-time method that estimates the 3D human pose directly in the popular Bio Vision Hierarchy (BVH) format, given estimations of the 2D body joints originating from monocular color images. Our contributions include: (a) A novel and compact 2D pose NSRM representation. (b) A human body orientation classifier and an ensemble of orientation-tuned neural networks that regress the 3D human pose by also allowing for the decomposition of the body to an upper and lower kinematic hierarchy. This permits the recovery of the human pose even in the case of significant occlusions. (c) An efficient Inverse Kinematics solver that refines the neural-network-based solution providing 3D human pose estimations that are consistent with the limb sizes of a target person (if known). All the above yield a 33% accuracy improvement on the Human 3.6 Million (H3.6M) dataset compared to the baseline method (MocapNET) while maintaining real-time performance
https://www.youtube.com/watch?v=Jgz1MRq-I-k
Other
850 stars 136 forks source link

Please, can you give us a video explaining how to install and run this program? #55

Closed 2655251820 closed 1 year ago

2655251820 commented 3 years ago

Because I don’t understand the code, and I don’t know how to install and configure the code running environment, you have explained it, but it is difficult for us who do not understand the code in non-English speaking countries to understand what you have done. If it is convenient, you can Can I explain this in a video? My friends and I are very interested, even though they don’t understand the code

2655251820 commented 3 years ago

And how to connect to Blender, this is our most concerned issue

AmmarkoV commented 3 years ago

Hello, thanks for opening the issue..! it is true that setting up the project can be daunting if you don't have experience with CMake etc.

First of all I will create a youtube video going over the setup of the project as you requested, however due to currently having various deadlines this will probably happen around June!

To create a realistic rendering of a human in blender you can use : A) Makehuman B) My CMU+Face Rig C) Blender D) The Makehuman plugin for Blender for Makehuman E) The Makehuman plugin for Blender for Blender F) The Makewalk plugin for Blender G) A BVH file, even if you cant get MocapNET to compile you can download and try some of the CMU BVH files ..

Using all these you can create a 3D avatar with the appropriate rigging, directly import it in blender and then using the makewalk plugin automatically retarget it with the BVH file..

However you can also open a BVH file directly with blender screen-2021-05-10-13-07-25 screen-2021-05-10-13-09-47

and then set up your own rigging to your model Unfortuntely I dont know your Blender/3D animation expertise but I think the above is a good overview to answer your questions

Hezhexi2002 commented 2 years ago

@AmmarkoV Hi,I'm a undergratuate from China and I have attended the Adaptive Computing Challenge 2021 with Xilinx and I want to use the KV260 AI starterkit to make a mocap which Iead me to find your project,I can‘t install your MocapNET directly using the initialize.sh script which shown in this picture: image I'm not sure if it is caused by the architecture of KV260,I checked the shell script and found it was mainly download some pretrained model and configure the environment of the platform.And the xilinx's vitis-ai repository have intergrated the openpose model,So I want to ask you how you finally generate the BVH files through MocapNET and I wonder if I can use the openpose model in the vitis library to achieve it too,I would appreciate it if you can give a reply:-)

AmmarkoV commented 2 years ago

I moved the issue of @Hezhexi2002 to a new issue so that it is not mixed with the making a video issue..

Hezhexi2002 commented 2 years ago

I moved the issue of @Hezhexi2002 to a new issue so that it is not mixed with the making a video issue..

OK,thanks a lot,I'm so sorry that I don't reply you in time because recently we have celebrated the Spring Festival in China,I will soon check the new issue and try it again:-)

socketing commented 2 years ago

我将用中文与你沟通,方便告诉我你的VX或者QQ吗,这样更方便我们交流。

socketing commented 2 years ago

@Hezhexi2002

Hezhexi2002 commented 2 years ago

我将用中文与你沟通,方便告诉我你的VX或者QQ吗,这样更方便我们交流。

用微信吧,主要是我也没有在本地跑起来过,但是我现在主要是想移植到kv260上mmqrcode1645967052419.png

AmmarkoV commented 1 year ago

Hello I added a video, so after a long while closing this issue :) !

YouTube Link