davidpagnon
commented
1 month ago Hi, thanks for you interest!
Here is what I did:
- Human being
- regular pipeline to obtain the trc file
- scale the model of the person
- Bike
- Trained a DeepLabCut model
- Converted the results to OpenPose json format
- Edited the CUSTOM model in Config.toml, in order to triangulate the DeepLabCut points
- Created an OpenSim model. I did it the old school way by creating stl files (Blender works, and so do many other pieces of softwares: search "create stl file" on Google) and then creating an .osim file with joints, bodies, and markers by hand. But you should check https://www.opensimcreator.com/, I have not tried it but it might be much more straightforward. Note that you don't have to have good-looking bodies, you could create a sphere or anything for each of them as long as their properties are right
- Scale the model you created from DeepLabCut. You will have to edit the Scaling_setup.xml file so that it takes the right keypoints.
- From that, you have two options :
- perform IK independently for both models
- Or if you want both person and deeplabcut models to be constrained to each other, you can assemble the trc files with this script: trc_combine.py, and then perform IK on both models together
More information here: https://github.com/perfanalytics/pose2sim?tab=readme-ov-file#with-deeplabcut
Great work ! In your trailer you show an OpenSim skeleton riding a bicycle. How do you integrate a handheld device in the model. Do you extend the OpenSim model with additional "bones" and "joints", or what would you recommend ?