Cannot get reliable simulation

[costashatz]

Hello,

Thanks for the awesome library.

I am having trouble simulating some characters with MuJoCo.

Here are the main issues:

• I get "weird" penetrations and reaction forces. See the following video for example (check the feet collisions):

https://user-images.githubusercontent.com/1283922/202186676-c452fa3d-247d-4520-8ee7-b1bbc8996f54.mp4

I have multi-ccd enabled (even with disabled, I get similar results) and custom solref/solimp for the contacts (with the defaults, I get even worse results --- simulation breaks).

• The simulation breaks too often (I get resets and bad values). I have attempted too many options (solvers, integration schemes, friction coefficients, joint friction, joint damping, joint stiffness, density, solref, solimp, disabling self-collisions [by hand], etc..) and I cannot find values that work reliably in most cases. I have scenes with humanoids, quadrupeds, simple objects, other biped characters etc. Is it expected that I have to tune all the parameters per scene/character to get something reliable? Or is there something that I am doing wrongly?
• The timings are "suprising". Replicating exactly the same scenes with PhysX, I get 2-10x the performance with no actual visual difference in the realism of the results (even if I do fine steps and look in detail --- obviously the results are different, but difficult to choose one from the other in terms of realism). This gets amplified if I start adding more characters: the more characters, the bigger the performance gap.

Any ideas on what I can try?

I am attaching one of my models for you to inspect in case you see something completely off or have any feedback to give.

model_manikin.zip

Thanks a lot, Konstantinos

[joaopfg]

Hey, I'm not a MuJoCo developer. I'm just a beginner user. But I had this kind of penetration problem in my simulation and the reason was that I was defining collision geometries that weren't fully covering the bodies in my model in the MJCF file. So, I would suggest you to look at your collision geometries.

[costashatz]

Hey @joaopfg! Thanks for replying and the interest. This is not the issue. I have only one mesh for each body: both visual and collision. I double-checked to be sure :) Also, if I decrease the timestep quite a bit, I get normal reaction forces. But this makes the simulation too slow for what I need it for.

[Balint-H]

Heya, non-MuJoCo team user here. Do you get the same issue with the standard humanoid using your physics settings? Also you may try substituting a primitive geom for collisions at the foot (while keeping the mesh for visuals).

[yuvaltassa]

There are lot of things wrong with this model, most importantly:

• (non free) joints always need a bit of damping. This is both realistic and better for integration.
• Don't collide with very thin boxes. Use thick boxes or planes.

These are the changes I made, and it now looks fine, though I haven't experimented extensively. If you clean up the model (use defaults, make z point up etc. I might be able to help further). Lines 135-136:

<!-- <joint name="manikin:rMarker_rGround_joint" type="free"/> -->
<geom name="manikin:rMarker_rGround_shape_0" size="2.2 0.15 2.2" pos="0 -0.015 0" type="box" friction="0.6 0.6 0.6" mass="34.56" rgba="1 0 0 1"/>

(why does your ground have a free joint??) Lines 3-6:

<default>
  <joint damping="0.1"/>
</default>
<option timestep="0.01" gravity="0 -9.82 0" magnetic="0 0 0" o_solref="0.01 1" o_solimp="0.9 0.95 0.001" integrator="implicit">

Adding more characters should have no effect on the physics, this is disturbing. I can't speak for physX, but it is very common for gaming physics simulators to silently add a bit of joint damping.

Also, @Balint-H's recommendation regarding the feet is a good one.

[costashatz]

@yuvaltassa Thanks for the feedback and thanks again for all the effort for the library.

• (non free) joints always need a bit of damping. This is both realistic and better for integration.

Don't I have damping to all joints? In my xml, all joints have "damping=0.2" (except the free joints of course).

Adding more characters should have no effect on the physics, this is disturbing. I can't speak for physX, but it is very common for gaming physics simulators to silently add a bit of joint damping.

What I meant is that the more characters I add, the slower the simulation is. For example, if I add 25 humanoids, on my PC it is hardly real-time. Is this something to be expected?

If you clean up the model (use defaults, make z point up etc. I might be able to help further)

Is z-axis up something "required" for MuJoCo to work as expected? Or should MuJoCo operate the same regardless of the axis of gravity?

(why does your ground have a free joint??)

This is complicated to explain, but I have a strong weld constraint to it, so that I can "animate" it. Could this be an issue?

• Don't collide with very thin boxes. Use thick boxes or planes. Lines 3-6:

Thanks for the suggestions. I'll check how I can incorporate those in my pipeline and get back to you.

[joaopfg]

@Balint-H @yuvaltassa Hooking on the the related discussion, I would like to ask some questions.

When I add two geometries to a body, will the first one always be used for visualization and the second one for collision? It looks like that in my tests. What if I add more than two geometries ?

Also, when I leave my model with only one geometry (meshes), I have the penetration problem. But when I add a second geometry (cylinders or capsules) covering the first geometry, I have no more collisions. Why does the problem appear with meshes ? is it because meshes create concave surfaces in some regions and makes MuJoCo's collision detector fail ? If so, could you give me an intuition on why it is difficult to detect collisions between concave surfaces ?

[yuvaltassa]

@costashatz

Don't I have damping to all joints? In my xml, all joints have "damping=0.2" (except the free joints of course).

Ah in that case sorry, I expect these things to be in defaults. But in this case, I'm surprised that adding my line did anything at all (since I actually decreased the damping?)

What I meant is that the more characters I add, the slower the simulation is. For example, if I add 25 humanoids, on my PC it is hardly real-time. Is this something to be expected?

Yes of course. How do imagine adding more characters to the scene would not increase running time? Here is a model with 22 humanoids, could serve as reference perhaps.

Is z-axis up something "required" for MuJoCo to work as expected? Or should MuJoCo operate the same regardless of the axis of gravity?

It's not "required", but it's difficult to examine models in simulate. Basically that assumption is build into the viewer, not the physics.

This is complicated to explain, but I have a strong weld constraint to it, so that I can "animate" it. Could this be an issue?

You should probably use a mocap body rather than a free joint with a weld.

@joaopfg,

will the first one always be used for visualization and the second one for collision?

No. Please read the relevant documentation.

@costashatz, @joaopfg,

Please examine the XMLs of the Menagerie models for examples of how to use defaults and make a collision primitive layer.

PS @costashatz, your models look pretty cool, maybe one day submit them to the Menagerie? Do you plan to add spatial tendons to control the manikins with strings? I've always wanted to make such a model :slightly_smiling_face:

[costashatz]

Ah in that case sorry, I expect these things to be in defaults. But in this case, I'm surprised that adding my line did anything at all (since I actually decreased the damping?)

From my tests, making the ground thicker (and resetting contact solver params) makes everything more stable.

Yes of course. How do imagine adding more characters to the scene would not increase running time? Here is a model with 22 humanoids, could serve as reference perhaps.

I meant to compare to PhysX here. Of course, adding more characters will increase the running time, but there seem to be like a linear factor to MuJoCo. Like adding 5 characters, makes it like 5x slower than 1 character (of course it depends on many factors, like how much all the characters interact with each other etc..). In any case, I'll take your test scene and make experiments and see if I am getting similar performance in my app. Thanks for pointing this out :smiley:

It's not "required", but it's difficult to examine models in simulate. Basically that assumption is build into the viewer, not the physics.

Good to know that MuJoCo should handle everything ok with any up-axis.

You should probably use a mocap body rather than a free joint with a weld.

Yeap. This is exactly what I am doing. Having the ground with a free joint and adding a mocap body attached to it with a weld constraint. Or what you are suggesting is that a mocap body can generate collisions. Can I use a mocap body to interact with the rest of the physics? From the docs, I assumed that a mocap body cannot have collision shapes or any interaction with the rest of the physics. So this was my trick to make this work: Ground has a free joint, I attach a mocap body to it with a strong weld so that I can say to the mocap body to follow whatever path I want and the ground follows but interacting with the rest of the world. Let me know if I am mistaken somewhere here.

Please examine the XMLs of the Menagerie models for examples of how to use defaults and make a collision primitive layer.

I am creating the model using the C++ api (under user folder -- I need to generate those models automatically on the fly) and just saving the outcome as an XML to share it here. If there are any tips & tricks for setting defaults in the mjCModel, please let me know.

PS @costashatz, your models look pretty cool, maybe one day submit them to the Menagerie?

I never thought of this actually. Might consider it as a weekend or Christmas project :+1:

Do you plan to add spatial tendons to control the manikins with strings? I've always wanted to make such a model

Hahah. Not at the moment, but seems like a nice addition to the model :smile:

[yuvaltassa]

Or what you are suggesting is that a mocap body can generate collisions. Can I use a mocap body to interact with the rest of the physics?

Yes.

[Balint-H]

Another way to increase stability of joints is to increase their armature parameter. (Although this won't help with collision/contact issues).

[costashatz]

Another way to increase stability of joints is to increase their armature parameter. (Although this won't help with collision/contact issues).

This makes everything look more damped and less "realistic". And yes the collision/contact issues would be still there..