IndexOutOfRangeException for importing mujoco_menagerie models into Unity

[shreyasgite]

IndexOutOfRangeException: The mesh exceeds the number of vertices per mesh allowed by Unity. (2728366986 > 65535)

Hi,

I'm using MuJoCo with unity plugin. And trying to import the mujoco_menagerie models. However Unity won't let me do it, and throws out following error message:

IndexOutOfRangeException: The mesh exceeds the number of vertices per mesh allowed by Unity. (2728366986 > 65535) Mujoco.StlMeshParser.ParseBinary (System.Byte[] stlFileContents, UnityEngine.Vector3 scale) (at /Users/shreyas/mujoco/unity/Editor/Importer/StlMeshParser.cs:64) Mujoco.MjImporterWithAssets.CopyMeshAndRescale (System.String sourceFilePath, System.String targetFilePath, UnityEngine.Vector3 scale) (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterWithAssets.cs:181) Mujoco.MjImporterWithAssets.ParseMesh (System.Xml.XmlElement parentNode) (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterWithAssets.cs:160) Mujoco.MjImporterWithAssets.ParseAssets (System.Xml.XmlElement parentNode) (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterWithAssets.cs:125) Mujoco.MjImporterWithAssets.ParseRoot (UnityEngine.GameObject parentObject, System.Xml.XmlElement parentNode) (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterWithAssets.cs:88) Mujoco.MjcfImporter.ImportXml (System.Xml.XmlDocument mjcfXml, System.String name) (at /Users/shreyas/mujoco/unity/Runtime/Importer/MjcfImporter.cs:100) UnityEngine.Debug:LogException(Exception) Mujoco.MjcfImporter:ImportXml(XmlDocument, String) (at /Users/shreyas/mujoco/unity/Runtime/Importer/MjcfImporter.cs:104) Mujoco.MjImporterWithAssets:ImportString(String, String, String) (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterWithAssets.cs:74) Mujoco.MjImporterWithAssets:ImportFile(String) (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterWithAssets.cs:59) Mujoco.MjImporterEditorWindow:Apply() (at /Users/shreyas/mujoco/unity/Editor/Importer/MjImporterEditorWindow.cs:28)

Is there a workaround for this? Thanks a lot!

[kevinzakka]

No Menagerie model has 2.7 billion vertices, this looks like a Unity-specific bug.

[yuvaltassa]

I'm suspecting some sort of underflow.

@shreyasgite, can you specify which model you are trying to import?

Also, Unity version, MuJoCo version, operating system, generally stuff required to reproduce.

[shreyasgite]

Sorry about not giving this info before. Thanks Kevin.

I am trying to import this into Unity to create simple robot learning sims from computer vision perspective.

The above error happens with all the mujoco_menagerie (Cassie, anymal, panda, unitree) models. But the import works for standard models form Mujoco library (humanoid, humanoid 100, tendon arm etc)

OS: macOS Monterey 12.6 Chip: Apple M1 Max Unity: 2021.3.10f1 Mujoco: 2.2.2

[pkrack]

Tried importing panda (and others from the menagerie) and got the same error (IndexOutOfRangeException: The mesh exceeds the number of vertices per mesh allowed by Unity. (somenumber > 65535))

Unity version: 2021.3.11.f1 MuJoCo version: 2.2.2 OS: Ubuntu 22.04

Steps to reproduce:

1. Import panda.xml from the menagerie

After adding a little print statement in CopyMeshAndRescale I discovered that the error occurs as soon as sourceFilePath, contains ~/Downloads/mujoco_menagerie-main (1)/franka_emika_panda/assets/link5_collision_0.obj.

The functions CopyMeshAndRescale / ParseBinary treat that obj file as a binary .stl, which results in reading "random" values here.

So I guess troubleshooting why an obj file gets fed into that function is the next step.

[pkrack]

It looks to me like the Unity plugin simply does not support importing from obj file, which the menagerie assets apparently use.

You can convert the .obj files to .stl files using the assimp command line tool from the Open Asset Import Library and change the references to the .obj file in the appropriate XML file (model_name.xml)

In the case of the panda.xml file I also had to remove the line with <option integrator="implicit">, as implicit appears to be unsupported.

Example to convert all the obj files into binary STL: for f in *.obj; do assimp export $f ${f%obj}stl -fstlb; done Example to change the xml file accordingly (in the panda case the assets are referenced in panda.xml, not in scene.: sed -i s/.obj/.stl/ panda.xml

Now the model imports (which takes a surprising amount of time where unity is frozen. just wait) but the textures are all ugly and most importantly it doesn't work: a bunch of PhysicsRuntimeException: BADQACC: NaN/inf in qacc. errors are thrown.

I guess it's time for a new ticket as the inital problem was fixed? Or changing the title to menagerie models don't work, although I don't know yet if it is the case for all the menagerie models.

EDIT: The model with the converted meshes works as expected in the MuJoCo simulator. The assimp conversion is not the problem.

[pkrack]

Got it all working.

1. Convert all .obj fils to binary STL (e.g. with assimp): for f in *.obj; do assimp export $f ${f%obj}stl -fstlb; done
2. Change the .obj references in model.xml: sed -i s/.obj/.stl/ model.xml (note: in <model_name>.xml not in world.xml
3. Import the model (takes a while)
4. In the case of panda: the xml <option integrator="implicit"> is not supported by Unity. Remove it. If there is an xml tag that is not supported the mujoco plugin will produce an appropriate when you try to import a model.
5. Disable the Mesh Renderer components of all collision meshes (look for gameObject with a MjGeom that has Filtering.Group set to 3
6. Enable CommonParameters.CtrlLimited and CommonParameters.ForceLimited on all actuator components.

[teamclouday]

@pkrack Thank you very much! I'm able to load panda.xml by following your steps.