search

ButzYung / SystemAnimatorOnline

XR Animator, AI-based Full Body Motion Capture and Extended Reality (XR) solution, powered by System Animator Online
https://sao.animetheme.com/XR_Animator.html
826 stars 69 forks source link

The original capture in the wireframe skeleton seems better than the processed for full body #63

Open PeterJared opened 8 months ago

PeterJared commented 8 months ago

The top body captures are systematically good, always. The full body captures, after trying many settings seems worse than the original wireframe capture, the black window with a wireframed skeleton that appears on the right. The original capture seems to stick very well to the floor when the processed one tilts, it is less steady and the feet keep less his place on the floor. Please, check the example: https://drive.google.com/file/d/1182AHu8aSup_Sj4ybFBHSniuXQu6k5mp/view?usp=drive_link

ButzYung commented 7 months ago

You link doesn't seem to be public.

PeterJared commented 7 months ago

Sorry, I changed the rights of the link Let me know if any problem https://drive.google.com/file/d/1182AHu8aSup_Sj4ybFBHSniuXQu6k5mp/view?usp=sharing

ButzYung commented 7 months ago

This looks fairly fine to me. The legs may be a bit too close together occasionally, but that's an issue of the default model itself as the resting poses of the legs of the default model are not totally straight and are bending a bit inward. If you use another model with straight legs the problem will be gone. Also it seems your webcam is placed at a relatively high position pointing slightly downward? This may explain the tilt issue, and the nature of the 3D perspective camera may also cause some visual difference from the 2D wireframe display.

PeterJared commented 7 months ago

My point is that the original capture from MediaPipe, which appears in the video, seems better. I guess there is a significant problem with calculating the pelvis position, but the feet do not drift in the original capture displayed by the software. The upper body has very good accuracy. I really like the app and am happy with the performance of the upper body alone, without any extras. I was wondering about the difference in precision, as it results in sliding feet in the animations.

ButzYung commented 7 months ago

The translation of MediaPipe mocap data to actual joint rotations on avatar model is a complicated process, especially when you have to take various differences between the source and avatar model such as body proportion into account. It's difficult to make them look exactly the same. Besides, the wireframe display is 2D. Even when things look stable in 2D doesn't mean there isn't any fluctuation in the third dimension. That's why the actual 3D view of the data doesn't look as stable as the 2D wireframe.

PeterJared commented 7 months ago

Thanks a lot to developer of this software, I think in overall is really good.