MuJoCo 3.0 - memory sparsification, contact islands and multi-threading, non-convex geometries, deformable bodies

[josephrocca]

Wondering if either yourself or @zalo have heard the MuJoCo 3.0 news? Pretty incredible new features (quoted from the release notes):

• memory sparsification: MuJoCo already uses sparse algebra for large models, but several key matrices use dense memory allocation, while touching only non-zero entries. As a result, very large models simply do not fit in available memory. We have completed the memory-compression refactor for the Conjugate Gradient solver, which reduced the memory footprint of a scene with a humanoid and 100 free objects by a factor of 80, and are almost done sparsifying the remaining algorithms.
• contact islands and multi-threading: MuJoCo 3 detects “contact islands”: sub-components of the scene which do not interact. For example in a scene with two robots, they are assigned to separate islands as long as the robots don’t touch each other. Islands can be solved-for independently, for example on separate threads. Using a new thread-pool API, designed to exploit such parallelization opportunities, a model with 22 humanoids runs 3x faster. Threading over contact islands is currently only implemented for the Conjugate Gradient solver, implementation for all other solvers is in progress. Other such intra-scene parallelization opportunities, for example collision detection, are also forthcoming.
• non-convex geometries: MuJoCo 3 adds support for collision geometries defined via signed distance functions (SDFs), allowing users to create new primitives by specifying the distance from any given location to the closest point on a surface. These geometries are not constrained to be convex. We provide examples of what is possible including tori, gears, nuts and bolts. Additionally, users can import any mesh and generate a multiresolution voxelized SDF at model compilation time. In contrast to meshes, SDFs allow the number of contacts to be independent from the mesh resolution, making the cost of collision detection cheaper and more predictable. MuJoCo 3 SDFs can collide with all existing MuJoCo primitives and meshes.
• deformable bodies: MuJoCo 3 introduces a new kind of deformable body called flex. These are collections of segments (1D), triangles (2D) and tetrahedra (3D), corresponding to rope, cloth and deformable volumetric shapes like biological tissue. Flex bodies are not defined in a hierarchical kinematic tree, and can therefore simulate closed loop structures of any topological genus like rubber bands and T-shirts. Deformation can be controlled either with equality constraints (for relatively stiff bodies) or with passive forces (for relatively soft bodies).

Would be awesome to play around with high-performance non-convex and deformable bodies in the browser 🤯

CC @lo-th

[zalo]

I'd be happy to add bindings for the new features/versions of MuJoCo, but I'm basically useless for generating new WASM compilations. 😅

[stillonearth]

I haven't merged https://github.com/stillonearth/MuJoCo-WASM/pull/3/files#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a back, but it seems there's CI work done to make it automatic.

[zalo]

@stillonearth I just finished a proper port of your 2.3.5 work: https://github.com/zalo/mujoco_wasm/compare/main...zalo:mujoco_wasm:upgrade_mujoco Seems like everything is good except PNG loading/textures don't work anymore 🤔

[stillonearth]

I also disabled textures previously, because wasm bindings don't handle rendering and I was unable to get them work with emscripten.

[zalo]

I also disabled textures previously, because wasm bindings don't handle rendering and I was unable to get them work with emscripten.

How were you able to get lodepng working in the original wasm compilation? Did something change in MuJoCo?

[Zedderrs]

Is there a MuJoCo 3.0 wasm that exists? If not, can someone help me how I can port the newest MuJoCo?

[stillonearth]

https://github.com/zalo/mujoco_wasm is up-to-date fork. 3.0 hasn't been ported yet.