Implement IK with URDF models

[jfslin]

I am trying to use rbdl-toolkit to perform IK, but have been unable to figure out how to proceed. I have separated this into 2 issues:

• Unable to load URDF models
• Unable to IK

Using the rbdl-toolkit + plugins precompiled as AppImage, I was able to enable all the plug-ins, and was able to load a .c3d file. However, I get a missing model error instead (likely related to the missing model issue in #13):

I was also wondering if there was an existing guide on how to use the IK plugin? Thank you.

[ju6ge]

So at the moment doing IK with urdf models is not supported. The reason for this is that in order to do IK you need TWO sets of markers. One set of ModelMarkers and one set of MocapMarkers. The ModelMarkers describe the position of each marker on the model meaning on which segment and at what position. The MocapMarkers describe the Marker frames or where each marker should be at a given point in time. Loading a c3d you should have the MocapMarkers loaded. So I probably need to work on the wording of the errors, because you are most certainly are missing the ModelMarkers. At this point rbdl-toolkit only tries to load these markers from lua models. This is because I am unaware of any urdf models having that information stored in its xml and therefor I have no example to go from also even if I did it is unclear to me if that would translate to more than that specific model since many things in urdf seam to be very custom to who ever created it for their specific purpose.

The good news is that it is not hard to add support. All the functionality like IK and even animations are implemented as rbdl-toolkit plugins / model extentions. If you look into the code of the MocapMarkerPlugin you will see a ModelMarkerExtention. This is the class that stores the ModelMarker data. The IK Plugin just looks if a given model has a MocapMarkerExtention and a ModelMarkerExtention loaded, and then uses the data from there. All you would need to do is fill a instance of the ModelMarkerExtention class with your markers and add that to your model. Then the IK plugin should work just fine.

To do this you can write a rbdl-toolkit plugin. There is a template so you do not have to do this from scratch. There is also a AppImage to build the plugin so that it links against the same library versions used by the released AppImage so that they work together and sparing you the effort of setting up the cmake environment correctly.

[jfslin]

Sure, I can build an extension for URDF. Do I need to roll back to Ubuntu 18.04 to ensure AppImage builds properly?

[jfslin]

Also, I was trying to IK with the Lua model, which gave me the error.

[ju6ge]

It still holds true that the ModelMarkers are missing from that model, because if you loaded the c3d the Markerframes are loaded. Take a look at the model i provided you in #12. You should not need to roll back to build a plugin that works with the AppImage itself, provided you use the pluginbuilder. I have build plugins with if from different Operating Systems that then correctly worked with the AppImage. But since it seams there is trouble with the AppImage itself running on Ubuntu 20.04 on your system, rolling back to 18.04 would still be best.

[jfslin]

So I think there's a few different stages with getting URDF IK working:

• Write a method to generate URDF modelmarker...either inside the URDF model file itself, or as a separate file
• Load a .c3d file and use its marker position at frame 0 to create model markers and denote which limb/frame each marker is attached to
• Add ability to modify the model's state (to a T-pose, for example), weither in the UI or from an external file, so the pose can modified for modelmarker attachment (ie similar to Puppeteer's demo video: https://www.youtube.com/watch?v=-BVTGs8xCnQ)

URDF model So @RVSagar and I are thinking of having two separate files for URDF models/modelmarkers, in the interest of keeping URDF files compatible with ROS. It might be a hassle to maintain both a URDF model file, as well as a URDF modelmarker file though.

Looking around the URDF definitions on ROS, there is this sensor tag that we can potentially use, though they do also note that "the sensor element has been implemented in the URDF Dom but has never really been used in application. This is a project that was dropped anyone is encouraged to pick it up and extend it to sensor hardware applications. Please contribute! "

Also, note on toolkit architecture that might help with the modifications: https://github.com/ORB-HD/rbdl-toolkit/issues/16#issuecomment-772551436