Isaac Sim 2023.1.1 crashes when starting simulation with provided robots

[TL-4319]

Hi, I have been testing the new release using the native installation instruction and the largescale environment is great!

I am running into an issue now where Isaac Sim crashes whenever I start a simulation with the provided robot USD (both Husky and Jackal). However, it works fine with an USD that i made attached here viper.zip

The error dump is attached below. 70aa3d0c-9bc5-4cea-d0b0a2ae-b5ed8441.dmp.zip

The error message that appeared is

2024-09-13 21:14:30 [49,900ms] [Warning] [omni.fabric.plugin] No source has valid data array=0x1e72dff0 usdValid=0 cpuValid=0
2024-09-13 21:14:30 [49,900ms] [Warning] [omni.fabric.plugin] No source has valid data array=0x1e731628 usdValid=0 cpuValid=0
terminate called without an active exception
2024-09-13 21:14:30 [0ms] [Warn] [carb.crashreporter-breakpad.plugin] [crash] A crash has occurred.  If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds.  This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.
2024-09-13 21:14:30 [47ms] [Warn] [carb.crashreporter-breakpad.plugin] [crash] compressing 3 files into '/home/lagerworkstation/.local/share/ov/data/Kit/Isaac-Sim/2023.1/9d6e91f4-19f3-485f-e17143b4-ead4777d.dmp.zip' for upload.
'crashreport.gui' exited with code 127Segmentation fault (core dumped)

I have brought up the issue on Nvidia forum but am curious if you guys encountered this problem as well?

Thanks for the work

[AntoineRichard]

That’s odd! Could you tell me what hardware you’re running? Did you check that ROS2 was sourced prior to running the simulation? Let me know, Cheers,

Antoine

[TL-4319]

Hi Antoine,

Here are the information of the workstation I am using:

• Intel Xeon(R) Gold 6244 with 16 cores
• 96 GB of RAM
• RTX3080 with 10 GB of VRAM

Workstation software:

• Ubuntu 22.04.4 LTS
• GNOME 42.9
• NVIDIA driver 535.183.01

I installed ROS2 using apt. I installed IsaacSim 2023.1.1 using the Omniverse Launcher with their tutorial. I installed OmniLRS v2 using the ReadMe.

Each terminal is sourced with ROS2 using

source /opt/ros/humble/setup.bash

To run IsaacSim, I also need to set these environment variables following suggestion in the terminal when starting IsaacSim I think to set the correct middleware layer

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/<username>/.local/share/ov/pkg/isaac-sim-2023.1.1/exts/omni.isaac.ros2_bridge/humble/lib
RMW_IMPLEMENTATION=rmw_fastrtps_cpp

Here's what I do to cause the crash. Starting an IsaacSim session using

~/.local/share/ov/pkg/isaac-sim-2023.1.1/isaac-sim.sh

Open an USD provided by OmniLRS v2 such as the husky. When I hit play, it crashes and result in the terminal message and file dump above which was not too clear.

Doing the same with the viper.USD does not cause any issue. This process also worked with the USD downloaded from Drive from the previous OmniLRS version.

[AntoineRichard]

The sourcing looks fine and the DDS settings look good as well. We did run the simulation on other systems without issues. It's interesting that this is happening. I know you need an active internet connection to start the sim. Maybe this is why?

I'm not overly fond of zip files. Could you instead dump the full output of the simulation as it crashes?

You should not need to press play. By default, the simulation should be running. Is it not?

The large scale environments will not work with Isaac 4.1.0. (it will be the renderer should do weird things) If you're interested, we could do a live debug session. Email me if you want to do that. Let me know,

Best,

Antoine

[TL-4319]

I do have a wired internet connection on the machine.

I recorded the behaviors as followed. Note that in the attached videos, a new terminal is started just to make sure nothing weird lingers between terminal session. The terminal output are logged via script

Test 1: Lunaryard 20m with the VIPER.usd robot that I made (works)

Screen capture

Terminal output

Test 2: Lunaryard 20m with the ros2_husky_PhysX_vlp16.usd robot downloaded from Drives (crashes)

Screen capture

Terminal output

Test 3: Empty scene (can load both viper.usd and ros2_husky_PhysX_vlp16.usd. Only ros2_husky_PhysX_vlp16.usd causes crash)

Screen capture

Terminal output

From these behavior, it might be something weird with the provided usd assets but I am not sure. The error message on the terminal was not clear and kinda only suggest users to investigate the dump file. This is the response from NVIDIA on the forum not sure about the cause and suggests Isaac Sim 4.1.0.

I appreciate the offer for the live debug session. I will test the docker installation first before setting up a session since I'm sure you are busy and would only prefer to use your time if only all other options are exhausted.

Best regards,

Tuan

[AntoineRichard]

I've had weird crashes in the past linked to the simulation not being "hot". I.e. it needed to be stepped quite a bit before adding a crazy amount of stuff in it. This week I'm flexible and commited to sort out issues with the sim, so really no bother at all. Hit me up at antoine.richard@uni.lu.

[TL-4319]

Nucleus needs to also be installed for assets to work