search

ARISE-Initiative / robosuite

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
https://robosuite.ai
Other
1.26k stars 404 forks source link

Cannot load elasticity plugin as defined in xml #439

Open sachinkundu opened 10 months ago

sachinkundu commented 10 months ago

I am trying to create a flexcomp object available in mujoco 3.

The XML definition is

<mujoco model="cloth">
    <statistic center=".4 0 .8" extent="1.3"/>

    <option wind="0 0 0" density="10" solver="CG" tolerance="1e-6">
    </option>

    <extension>
        <plugin plugin="mujoco.elasticity.shell"/>
    </extension>

    <worldbody>
        <camera name="top" pos="0.071 -0.018 2.756" xyaxes="0.019 -1.000 0.000 0.998 0.019 0.054"/>
        <camera name="fixed" mode="track" pos="-1.787 -0.875 1.327" xyaxes="0.442 -0.897 0.000 0.476 0.235 0.848"/>
        <geom name="floor" type="plane" size="0 0 .1"/>
        <light diffuse=".6 .6 .6" specular="0.2 0.2 0.2" pos="0 0 4" dir="0 0 -1"/>
        <body>
            <body name="object" pos="0 0 0.75">
                <flexcomp type="grid" count="9 19 1" spacing=".05 .05 .05" mass="10"
                          name="flag" radius="0.001">
                    <edge equality="true" damping="0.001"/>
                    <plugin plugin="mujoco.elasticity.shell">
                        <config key="poisson" value="0"/>
                        <config key="thickness" value="1e-2"/>
                        <!--Units are in Pa (SI)-->
                        <config key="young" value="3e6"/>
                    </plugin>
                </flexcomp>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 -0.045" name="bottom_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 0.03" name="top_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0.03 0.03 0" name="horizontal_radius_site"/>
            </body>
        </body>
    </worldbody>
</mujoco>

I can load this xml in mujoco simulate stand alone application. As you can see it uses engine plugin. When running simulate I have to put libelasticity.so in mujoco_plugin folder next to the application simulate.

I define a ClothObject 

class ClothObject(MujocoXMLObject):
    """
    Cloth object
    """

    def __init__(self, path, name):
        super().__init__(path, name=name, obj_type="collision", duplicate_collision_geoms=False)

and create this object as part of my environment initialization and add the object to the environment's placement initializer.

    def _create_cloth(self):
        self.cloth = ClothObject(str((Path(self.asset_path) / "cloth.xml").resolve()), "cloth")

During the run, after MujocoEnv._load_model() is run, a valid looking xml is generated(with my flexcomp cloth definition included). However this xml is not good enough for loading the sim as it fails with error

ValueError: Error: unrecognized plugin 'mujoco.elasticity.shell

There is plugin/libelasticity.so in /miniconda3/envs/robosuite/lib/python3.8/site-packages/mujoco and _load_all_bundled_plugins() in __init__.py of mujoco loads the plugin without an error.

The same xml and running test_load_plugin in bindings_test.py in /miniconda3/envs/robosuite/lib/python3.8/site-packages/mujoco also runs fine.

This makes me think that somehow robosuite run is changing the library loading path but I cannot seem to figure out where.

How can I load the engine plugins when running robosuite with mujoco 3?

sachinkundu commented 10 months ago

I have been debuggin a bit and surely there is something funky with loading an xml with plugins with robosuite.

Here is a simple script

import mujoco

MY_TEST_XML = r"""
<mujoco model="cloth">
    <statistic center=".4 0 .8" extent="1.3"/>

    <option wind="0 0 0" density="10" solver="CG" tolerance="1e-6">
    </option>

    <extension>
        <plugin plugin="mujoco.elasticity.shell"/>
    </extension>

    <worldbody>
        <camera name="top" pos="0.071 -0.018 2.756" xyaxes="0.019 -1.000 0.000 0.998 0.019 0.054"/>
        <camera name="fixed" mode="track" pos="-1.787 -0.875 1.327" xyaxes="0.442 -0.897 0.000 0.476 0.235 0.848"/>
        <geom name="floor" type="plane" size="0 0 .1"/>
        <light diffuse=".6 .6 .6" specular="0.2 0.2 0.2" pos="0 0 4" dir="0 0 -1"/>
        <body>
            <body name="object" pos="0 0 0.75">
                <flexcomp type="grid" count="9 19 1" spacing=".05 .05 .05" mass="10"
                          name="flag" radius="0.001">
                    <edge equality="true" damping="0.001"/>
                    <plugin plugin="mujoco.elasticity.shell">
                        <config key="poisson" value="0"/>
                        <config key="thickness" value="1e-2"/>
                        <!--Units are in Pa (SI)-->
                        <config key="young" value="3e6"/>
                    </plugin>
                </flexcomp>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 -0.045" name="bottom_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 0.03" name="top_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0.03 0.03 0" name="horizontal_radius_site"/>
            </body>
        </body>
    </worldbody>
</mujoco>
"""

model = mujoco.MjModel.from_xml_string(MY_TEST_XML)
data = mujoco.MjData(model)

which works.

I put some debug statements in self compiled mujoco binary(and replacing this in conda env for robosuite site-packages) and I can see that in void mjCModel::ResolvePlugin in user_model.cc active_plugins.size() == 1. However when running robosuite with the same xml(as part of full xml spec) active_plugins.size() == 0 which then errors out with ValueError: Error: unrecognized plugin 'mujoco.elasticity.shell'

Function void mjXReader::Extension(XMLElement* section) at Line 2548 in xml_native_reader.cc has a line ReadAttrTxt(elem, "plugin", plugin_name, /* required = */ true); which sets plugin state as true. This function is processed as part of mujoco run but does not get hit when running robosuite.

This means that somehow robosuite generates the xml in some other manner and hence bypasses the plugin loading logic of mujoco.

I am now lost as how to get robosuite working with plugins

sachinkundu commented 10 months ago

Should have noticed this before but it seems the generated xml which gets sent to mujoco.MjModel.from_xml_string does not have 

<extension>
        <plugin plugin="mujoco.elasticity.shell"/>
</extension>

which would explain why mujoco never loads the plugin.