Implement complex muscle models

[ShravanTata]

Currently mujoco only supports rigid tendon muscle models. While this is great to reduce complexity and improve speed, going further it would be great to include more complex muscle models. The suggested way to do this according to the documentation is to use the callback functions to rewrite the desired muscle dynamics. A feature missing right now is that muscle dynamics is assumed to be stateless. For elastic tendons, it is necessary to integrate the velocity of muscle model to update the states.

A solution would be to allow for a flag to indicate that the muscle model returns a state and needs to be integrated similar to the muscle activation dynamics.

This would allow for an including many more muscle models natively into the framework

[yuvaltassa]

Stateful MuJoCo actuators are currently assumed to have exactly one state, and this assumption will be difficult to change, though not impossible.

One simple solution, if I correctly understand what you want, is to add the mechanical dof directly. Instead of connecting the MuJoCo tendon directly to a bone, one could connect it to a small body attached to the bone with a single slider joint colinear with the tendon, acting as a passively elastic biomechanical tendon. This slider joint (1 position and 1 velocity) would probably have high stiffness and a bit of armature (representing internal inertia of the biomenchanical muscle), and its dynamics would correspond to those of an elastic tendon in series with the muscle.

[ShravanTata]

The solution with the single slider joint might not work. I haven't tried it but I don't think it would correspond to the exact muscle dynamics the hill-type muscle formulation would represent.

The current muscle implementation in MuJoCo reflects the model described in section 2.4 from this paper Millard et.al 2013 which complies with current MuJoCo assumption of actuators strictly having one state. What I am proposing is to be able to simulate muscles with elastic tendon formulation which is described in section 2.3 of the same paper. This involves integrating a state variable to obtain the actuator force.

For now, I can have an external integrator that gets called at every time step method call from MuJoCo and returns the appropriate muscle force as a gain through the callback function. This would work but it better to integrate the muscle dynamics with the internal integration schemes.

This feature would be beneficial for long term future to incorporate more complex bio-mechanical animal models

[yuvaltassa]

I completely agree with this statement.

However since while this is possible, it is not easy, we would first like to see more uptake from the biomechanics community, e.g. more models converted with @aikkala's OpenSim -> MuJoCo converter and used in research, before we commit resources to improving muscle modeling.

We do very much appreciate your input, and will post on this thread with any future developments.

[HuaweiWang]

@ShravanTata @yuvaltassa Very interesting discussion!

We actually produced some MSK models in MuJoCo based @aikkala's converting tool, but also optimized the moment arm and muscle force properties. One thing we found out was that similar force-length relationships can be generated in the current MuJoCo setup compared to muscle models that have elastic tendons (OpenSim). Details can be seen the chapter 4 [validation].

Indeed, without elastic tendons, some neural controllers are not easy to apply. For example, spinal reflexes, in which the major inputs are the muscle fiber length/velocity (instead of the full muscle-tendon unit).

The idea of adding a small body is very interesting and worth trying indeed. However, one concern might be how to automatically find the equilibrium point for the small body, especially at the beginning. If unreasonable position/velocity are given as the initial condition, dramatic movements will generate when the simulation starts.

[yuvaltassa]

Hi @ShravanTata

MuJoCo now supports multiple activation variables per actuator.

In this test you can see an example of a user-defined dynamics callback that implements 2nd-order dynamics.

If you get around to writing your muscle dynamics in C, please consider submitting a PR.

[ShravanTata]

Hi @yuvaltassa !

This is very exciting. Thanks for adding this feature.

I will checkout the test example and submit a PR with muscle dynamics that model elastic tendons

[ShravanTata]

Hi @yuvaltassa,

I am coming back to implementing this issue. I currently have multiple variations of simple and complex muscle models implemented in Cython and Python as a standalone library. I would like to interface this with MuJoCo. From the docs, the way to go about it would be write it as a plugin. As mentioned in the docs, it is suggested that I reach to the dev team before I proceed in this direction. Looking forward to discussions to implement this feature.

As mentioned above my current implementation is in Cython with Python interface. It would be useful to discuss if there is a way to use the already implemented library instead of porting it to C/C++. Thank you

[yuvaltassa]

Unfortunately no. I mean it might be possible to technically do this with Cython but in that case we would not be able to add your code to the repo. So ya, transcribe to C++ and send us a PR, we'll discuss details in the PR. If you wish you can make a very minimal PR that has only some of the features already implemented and we can discuss basic stuff like names, file locations etc.

[saran-t]

On the other hand the point of the plugin API is to allow features like this to be maintained separately from the MuJoCo codebase.

If you want, you could just wrap your Cython up into a plugin in a separate repo that you own, and ship your own wheels that people can install alongside MuJoCo.

[yuvaltassa]

Ya I was (possibly mistakenly) assuming the OP wants their muscle model in the main repo...

[ShravanTata]

@yuvaltassa I was open to both options with a preference to keep the muscle model independent from the main repo. I would appreciate any help towards setting up the plugin API as mentioned by @saran-t so that I don't have to rewrite the existing code. Let me know. Thanks

[yuvaltassa]

Hi @ShravanTata

There are several first party plugins in the plugin/ directory. I think in terms of code structure they should be sufficient. I believe @nimrod-gileadi is working on a first-party actuator plugin which might be even more relevant to your use case.

[yuvaltassa]

That said, I'm asking myself if what you are planning is really what you want. Muscle-tendon dynamics are mechanical, not neuro-chemical. Modeling them as proper dofs really is the correct thing. The fact that one can find equations in the literature that describe the muscle-tendon system does not mean that this is the correct way to go about it.

If your goal is to have a MuJoCo plugin that implements some model from the literature then you are doing the right thing. If your goal is to have the best model of a musculoskeletal system then a strong case can be made that you are not.

[ShravanTata]

@yuvaltassa Yes, my goal is to have a MuJoCo plugin that will implement several different muscle-tendon dynamics already described in the literature. Since this is the case it's good to know that the plugin approach is correct approach here.

I am curious about the last point you make. What and how would one describe a "best" musculoskeletal model in this context. Would be very interesting if you could share some of your thoughts here or point to me a discussion if you have already documented it elsewhere.

[yuvaltassa]

My point is that the muscle-tendon coupling is mechanical and should be modeled as such, see my first reply in this thread ☝️

[ShravanTata]

Thanks for the clarification :)