search

Jeff-sjtu / NIKI

Code of "NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation", CVPR 2023
269 stars 17 forks source link

Mixed results #17

Open drywet opened 1 year ago

drywet commented 1 year ago

Tested on some images from Yoga-82 dataset with the default demo.py script. The model seems too constrained by IK on this dataset, and sometimes it incorrectly detects a person.

What is the easiest way to improve results for this kind of pictures? Does it require fine-tuning the IK network, or can the IK constraints be adjusted in some config?

Bad results:
image-000001 image-000003 image-000006 image-000008
image-000010 image-000016



Ok results:
image-000002 image-000004 image-000005 image-000007
image-000009 image-000011 image-000012 image-000013
image-000014 image-000015
biansy000 commented 1 year ago

The human poses in current datasets are not diverse enough, so the network fails in Yoga poses. You may finetune the HRNet backbone on Bedlam and other hard datasets and then retrain the IK network...

drywet commented 1 year ago

Thank you for the explanation, I may try doing that soon or later, and also test Bedlam's solution