Scale in OpenSim

[carlosedubarreto]

First I must thank you a lot for this project, it made possible to have the most amazing mocap data possible, and I'm trying for about one year and a half.

And I'm making some changes to use it with mediapipe, so I could pack it all to use in blender. And I made the changes needed to make it work with your tools, but I dont know if I have to create another scale file to suit the markers place of mediapipe.

I know its not with you, since the markers are configurations on opensim, but, do you have a direction to point so I could do more research about it? I googled about it, but its very difficult to find information explaining stuff in opensim.

thanks a lot again for making my dream possible (my dream was to make mocap data from cameras 😃)

[davidpagnon]

Thank you for your kind words! Embedding this whole thing in Blender would be trully awesome. I was thinking of doing it with Maya (see here) since I am more familiar with its python wrapper, but I was thinking of migrating to Blender. This is off-topic, but do you already have a tool to import OpenSim .osim models and .mot kinematic files? If so, I'd be interested.

Anyway, just to make sure I understand correctly your issue: you want the markers on the OpenSim model to be consistent with mediapipe triangulated keypoints, rather than with OpenPose's? If so, there are three stages to it:

• Edit Pose2Sim/skeletons.py to have the right marker names and hierarchy.
• Adjust markers location on the OpenSim model (use GUI, and check <MarkerSet name="markerset"> section in .osim file). I personally had marker-based data with joint centers, that I could compare with the position of triangulated OpenPose keypoints (which are NOT at joint centers' positions).
• Then you will have to adjust the scale AND Inverse kinematics setup files. Change "Scale factors tab" in scaling, and tinker with weights in inverse kinematics. Are you familiar with OpenSim?

See Figure 1 on article: https://www.mdpi.com/1424-8220/22/7/2712/htm I did it on Maya, but Blender would work equally well.

[carlosedubarreto]

Great, you answered what I was needing. Thanks a lot. I'll try your instructions on the weekend...

Thanks a lot!!!

And answering the "off-topic" questions LOL. I did a version to import the motion in blender.

I made a test using 2 cameras(people seemed to like it a lot) https://www.patreon.com/posts/new-markerless-67368806

And I did the import from a record a friend did (he was the reason I've got to know your work) the result in blender after retargeting is here. (I didnt do any cleanup, it was all done by your filtering script and opensim) https://twitter.com/carlosedubarret/status/1533045782982578176

The material about the blender import can be seem here(i did a post in my patreon with all the files and instruction needed): https://www.patreon.com/posts/new-markerless-67368806

[carlosedubarreto]

And to import .osim file in blender i used another addon https://github.com/JonathanCamargo/BlendOsim

But my approach was using blender armatures, it was better in my opinion.

And I was able to import the mot file, but the mot file as basically angles in degrees, and making it work properly in blender was a pain in the but, and I gave up. I imported the data, but I felt I didnt know how to use the order of angles properly. I dont know how to explain better, but if I had to import just one angle per part of the body, it works right, but if I had to import 2 or 3 angles, things get messed up.

So my approach was using the location of the marker. and to do that I had to enable one optin in the inverse knematics xml file.

the change is explaned here https://simtk.org/plugins/phpBB/viewtopicPhpbb.php?f=91&t=13422&p=38657&start=0&view=

basically is to add the option <report_marker_locations>true</report_marker_locations> and when running the inverse knimatics, it will create a .sto file with the location of the markers (i guess its the markers LOL)