Finger problems, max/min angles?

[Aurosutru]

Using plugin version 1.60 (from main branch) in Blender 3.3.1, which is stable. The 2D output from mediapipe looks accurate but the conversion to 3D after transferring the animation has finger angles that are way off. Adjusting the finger bones in the Bone Constraint Properties tab using Before or After Original (Full) Mix helps with some bones, but they return to odd angles in other frames. As an example, in the picture below the right hand has fingers with adjustments, and the left hand is unadjusted. Any suggestions for further improving the capture? I suppose FreeMocap is an option, but I still wonder whether a single camera might work well, even for fingers.

Looked for the Blender panel that has Driver Remapping Value Mapping with from min, from max, etc. mentioned in Readme.md but cannot find it. Where exactly is that panel? Maybe this will need to wait until the next tutorial makes remapping clearer.

[KaijuKoder]

@Aurosutru, I'm having similar issues with fingers too. Some of the bone constraints added by BlendARmocap are problematic (not just fingers). Unfortunately I haven't been able to figure out the correct values, but the remapping panel you are looking for is on the driver, not the bones:

I spent several weeks experimenting with Freemocap and BlendARmocap. Both show promise, but they're not quite there yet.

[Aurosutru]

Thanks Kaiju. I'll look into these driver constraint settings. Am delving into FreeMocap too, with the hope that second and perhaps third cameras will work better than a single viewpoint.

[Aurosutru]

Cleaning up stray transfer data in my test file gave better transfer results. Tweaking of finger angle min/max settings is still needed when the hands make fists. Another problem is the swapping of middle and index fingers on both hands, which affects not only the nomenclature but the animation assignments too.

[cgtinker]

better

Changing mix/max setting may be required for making a proper fist, yea. Not sure if that's possible without any manual work to get always right (different finger length, base angles etc). I'm confused about the swapping? What's the reasoning behind that?

[Aurosutru]

In another issue comment you mentioned that A-pose gives better results than T-pose under some conditions. Is the starting pose problem the reason that default assignment of universal workable base angles, etc. is not possible. Would be nice it that worked out of the box by using a workable rest pose if that were all that is necessary.

The index/middle fingers swaps actually appears to be in the rigify rig, where the finger bones are mislabeled. BlendArMocap applies, for example, its index finger animations to the finger bones labeled index, but they are actually the middle finger bones according to the middle position in the hand.

[cgtinker]

In another issue comment you mentioned that A-pose gives better results than T-pose under some conditions. Is the starting pose problem the reason that default assignment of universal workable base angles, etc. is not possible. Would be nice it that worked out of the box by using a workable rest pose if that were all that is necessary.

Yes that might work, but that's something for the future. Really got to release this soonish as a lot has changed. Currently those offsets have to be applied manually, but in the future it might be an option to have something like a recalculate offsets button.

The index/middle fingers swaps actually appears to be in the rigify rig, where the finger bones are mislabeled. BlendArMocap applies, for example, its index finger animations to the finger bones labeled index, but they are actually the middle finger bones according to the middle position in the hand.

... this is new. not sure if that's on my side, I think rigify should fix that.

[Aurosutru]

I checked a fresh rigify rig in Blender 3.3.1 and the index and middle fingers are AOK. My working rig must be from an old, not so functional version of Blender.

[cgtinker]

aight, thanks for the input - I'll close this down as the issue seems to be fixed. Will release the current version next week and apply some minor fixes to the mapping before doing so. Planning to create a proper tutorial and proper docs which will prolly solve most of the questions.