[bug] Isaac lab streaming not working with client 103.1.1

[ben-kaye]

Describe the bug

omni.isaac.lab.app.AppLauncher does not work with livestream=1 using container

Steps to reproduce

run any Isaac lab script eg. ./isaaclab.sh -p source/standalone/tutorials/00_sim/create_empty.py --livestream 1 attempt to connect via streaming client fails unknown error 0x100

<ON HOST>
/workspace/isaaclab# ./isaaclab.sh -p source/standalone/tutorials/00_sim/create_empty.py --livestream 1
...
[5.919s] Simulation App Starting
[9.191s] app ready
2024-09-06 13:40:35 [9,560ms] [Warning] [omni.kit.imgui_renderer.plugin] _createExtendCursor: No windowing.
2024-09-06 13:40:35 [9,560ms] [Warning] [omni.kit.imgui_renderer.plugin] _createExtendCursor: No windowing.
[10.414s] Simulation App Startup Complete
[10.430s] [ext: omni.kit.streamsdk.plugins-4.5.1] startup
[10.439s] [ext: omni.kit.livestream.core-4.3.6] startup
[10.452s] [ext: omni.kit.livestream.native-4.1.0] startup

Failed to open [/var/run/utmp]
Active user not found. Using default user [kiosk]Streaming server started.
[10.841s] [ext: omni.physx.fabric-106.0.20] startup
[INFO]: Setup complete...

<CLICK CONNECT FROM STREAMING CLIENT ON CLIENT MACHINE>
main: thread_init: already added for thread
main: thread_init: already added for thread
main: thread_init: already added for thread
2024-09-06 13:40:41 [16,423ms] [Warning] [carb.livestream.plugin] onClientEventRaised: NVST_CCE_DISCONNECTED when m_connectionCount 0 != 1

I was able to fix this issue by reverting plugin versions.. see fix section

System Info

Describe the characteristic of your environment: HOST MACHINE

• Commit: tag:v1.1.0
• Isaac Sim Version: [e.g. 2022.2.0, this can be obtained by cat ${ISAACSIM_PATH}/VERSION]
• OS: Ubuntu 20.04
• GPU: 2 RTX4090
• CUDA: 12.2
• GPU Driver: 535.183.01

CLIENT

• GPU: TITAN X
• OS: ubuntu 20.04
• CUDA: 12.2
• Driver: 535.183.01
• Omniverse launcher: Streaming client 103.1.1

FIX

omni.isaac.lab.app.AppLauncher Line 564: replacing the plugin versions with versions in the working livestream.py example (_isaac_sim/standalone_examples/api/omni.isaac.kit/livestream.py) allows connections.

# Default App: Streaming Client from the Omniverse Launcher
# enable_extension("omni.kit.streamsdk.plugins-4.5.1")
# enable_extension("omni.kit.livestream.core-4.3.6")
# enable_extension("omni.kit.livestream.native-4.1.0")

enable_extension("omni.kit.streamsdk.plugins-3.2.1")
enable_extension("omni.kit.livestream.core-3.2.0")
enable_extension("omni.kit.livestream.native")

Checklist

• [X] I have checked that there is no similar issue in the repo (required)
• [X] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

• [ ] Isaac lab AppLauncher uses a working streaming configuration
• [ ]

[alberthli]

I get this error when trying to stream the headless native app in a docker container.

# start the container using the following command
# see: https://docs.omniverse.nvidia.com/isaacsim/latest/installation/install_container.html#isaac-sim-app-install-container
docker run --name isaac-sim --entrypoint bash -it --runtime=nvidia --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v ~/docker/isaac-sim/documents:/root/Documents:rw \
    nvcr.io/nvidia/isaac-sim:4.1.0

# within the container, start the headless native app
./runheadless.native.sh

The app starts correctly with output:

Loading user config located at: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/user.config.json'
[Info] [carb] Logging to file: /root/.nvidia-omniverse/logs/Kit/Isaac-Sim/4.1/kit_20240913_220521.log
2024-09-13 22:05:21 [3ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/6a8ac5f2-49e5-4c31-c93e0f88-8c741cc6.dmp.zip'
2024-09-13 22:05:21 [7ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/f8fd6a3b-c40d-4629-74075a8f-cf5f7d7e.dmp.zip'
2024-09-13 22:05:21 [9ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/be7be3d8-412f-4529-db1aaba4-f66bb6b5.dmp.zip'
2024-09-13 22:05:21 [10ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/2ce0020e-02fd-4192-0aee5d8c-eaf1c07f.dmp.zip'
2024-09-13 22:05:21 [25ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/8af0efb9-d788-4e4f-4a9b48a3-68d519c9.dmp.zip'
2024-09-13 22:05:21 [34ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.1/30d540e3-2a0c-4bd5-084f639f-e345cb8c.dmp.zip'
[0.534s] [ext: omni.kit.async_engine-0.0.0] startup
[1.342s] [ext: omni.stats-1.0.1] startup
[1.343s] [ext: omni.client-1.1.0] startup
[1.377s] [ext: omni.datastore-0.0.0] startup
[1.377s] [ext: omni.blobkey-1.1.0] startup
[1.378s] [ext: omni.ujitso.default-1.0.0] startup
[1.379s] [ext: omni.hsscclient-0.0.0] startup
[1.380s] [ext: omni.rtx.shadercache.vulkan-1.0.0] startup
[1.381s] [ext: omni.assets.plugins-0.0.0] startup
[1.383s] [ext: omni.gpu_foundation-0.0.0] startup
[1.401s] [ext: carb.windowing.plugins-1.0.0] startup
2024-09-13 22:05:23 [1,372ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:23 [1,372ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[1.402s] [ext: omni.kit.renderer.init-0.0.0] startup
2024-09-13 22:05:23 [1,462ms] [Warning] [omni.platforminfo.plugin] failed to open the default display.  Can't verify X Server version.
2024-09-13 22:05:24 [2,767ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:24 [2,767ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-09-13 22:05:24 [2,767ms] [Error] [carb.glinterop.plugin] GLInteropContext::init: carb::windowing is not available

|---------------------------------------------------------------------------------------------|
| Driver Version: 550.107.02    | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA H100 NVL                  | Yes: 0 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | 82969629.. |
|     |                                  |        |     |            | 1         |            |
|---------------------------------------------------------------------------------------------|
| 1   | NVIDIA H100 NVL                  | Yes: 1 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | 2a5af47c.. |
|     |                                  |        |     |            | 21        |            |
|---------------------------------------------------------------------------------------------|
| 2   | NVIDIA H100 NVL                  | Yes: 2 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | ba4067c8.. |
|     |                                  |        |     |            | 41        |            |
|---------------------------------------------------------------------------------------------|
| 3   | NVIDIA H100 NVL                  | Yes: 3 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | c97fab96.. |
|     |                                  |        |     |            | 61        |            |
|---------------------------------------------------------------------------------------------|
| 4   | NVIDIA H100 NVL                  | Yes: 4 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | e6c41be8.. |
|     |                                  |        |     |            | 81        |            |
|---------------------------------------------------------------------------------------------|
| 5   | NVIDIA H100 NVL                  | Yes: 5 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | 93fbd922.. |
|     |                                  |        |     |            | a1        |            |
|---------------------------------------------------------------------------------------------|
| 6   | NVIDIA H100 NVL                  | Yes: 6 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | 113faf8e.. |
|     |                                  |        |     |            | c1        |            |
|---------------------------------------------------------------------------------------------|
| 7   | NVIDIA H100 NVL                  | Yes: 7 |     | 95830   MB | 10de      | 0          |
|     |                                  |        |     |            | 2321      | c1f0d0c4.. |
|     |                                  |        |     |            | e1        |            |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 5.15.0-121-generic
| Processor: AMD EPYC 9354 32-Core Processor | Cores: 64 | Logical: 128
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 1547840 | Free Memory: 1531951
| Total Page/Swap (MB): 8191 | Free Page/Swap: 8191
|---------------------------------------------------------------------------------------------|
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 0
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 1
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 2
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 3
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 4
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 5
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 6
2024-09-13 22:05:38 [16,450ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 7
2024-09-13 22:05:38 [16,456ms] [Warning] [gpu.foundation.plugin] IOMMU is enabled.
[18.411s] [ext: omni.kit.pipapi-0.0.0] startup
[18.417s] [ext: omni.kit.pip_archive-0.0.0] startup
[18.418s] [ext: omni.pip.compute-1.4.0] startup
[18.419s] [ext: omni.pip.cloud-1.1.3] startup
[18.435s] [ext: omni.isaac.core_archive-2.3.0] startup
[18.436s] [ext: omni.isaac.ml_archive-2.0.1] startup
[18.436s] [ext: omni.mtlx-0.1.0] startup
[18.437s] [ext: omni.usd.config-1.0.4] startup
[18.444s] [ext: omni.gpucompute.plugins-0.0.0] startup
[18.444s] [ext: omni.usd.libs-1.0.1] startup
[18.531s] [ext: omni.kit.telemetry-0.5.0] startup
[18.641s] [ext: omni.kit.loop-isaac-1.2.0] startup
[18.644s] [ext: omni.kit.test-0.0.0] startup
[18.933s] [ext: omni.appwindow-1.1.8] startup
2024-09-13 22:05:40 [18,905ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:40 [18,905ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[18.938s] [ext: omni.kit.renderer.core-1.0.1] startup
2024-09-13 22:05:40 [18,926ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:40 [18,926ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-09-13 22:05:40 [18,994ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:40 [18,994ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[19.026s] [ext: omni.kit.renderer.capture-0.0.0] startup
[19.029s] [ext: omni.kit.renderer.imgui-1.0.1] startup
2024-09-13 22:05:40 [19,003ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:40 [19,003ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-09-13 22:05:40 [19,004ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:40 [19,004ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-09-13 22:05:40 [19,005ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:40 [19,005ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[19.150s] [ext: omni.ui-2.23.11] startup
[19.168s] [ext: omni.kit.mainwindow-1.0.3] startup
[19.171s] [ext: carb.audio-0.1.0] startup
[19.196s] [ext: omni.uiaudio-1.0.0] startup
[19.198s] [ext: omni.kit.uiapp-0.0.0] startup
[19.198s] [ext: omni.usd.schema.metrics.assembler-106.0.1] startup
[19.320s] [ext: omni.anim.graph.schema-106.0.2] startup
[19.328s] [ext: omni.usd.schema.audio-0.0.0] startup
[19.348s] [ext: omni.usd.schema.anim-0.0.0] startup
[19.372s] [ext: omni.usd.schema.semantics-0.0.0] startup
[19.377s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[19.384s] [ext: omni.usd.schema.forcefield-106.0.20] startup
[19.411s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[19.414s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[19.419s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[19.423s] [ext: omni.activity.core-1.0.1] startup
[19.425s] [ext: omni.usd_resolver-1.0.0] startup
[19.432s] [ext: omni.kit.window.popup_dialog-2.0.24] startup
[19.445s] [ext: omni.resourcemonitor-105.0.1] startup
[19.448s] [ext: omni.kit.actions.core-1.0.0] startup
[19.450s] [ext: omni.usd.core-1.2.11] startup
[19.454s] [ext: omni.kit.widget.nucleus_connector-1.1.8] startup
[19.462s] [ext: omni.graph.exec-0.9.3] startup
[19.463s] [ext: usdrt.scenegraph-7.4.8] startup
[19.513s] [ext: omni.kit.usd_undo-0.1.8] startup
[19.515s] [ext: omni.kit.exec.core-0.13.2] startup
[19.518s] [ext: omni.hydra.usdrt_delegate-7.4.7] startup
[19.538s] [ext: omni.timeline-1.0.10] startup
[19.540s] [ext: omni.kit.commands-1.4.9] startup
[19.552s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[19.554s] [ext: omni.hydra.scene_delegate-0.3.3] startup
[19.560s] [ext: omni.iray.libs-0.0.0] startup
[19.566s] [ext: omni.usd-1.11.2] startup
[19.661s] [ext: omni.mdl.neuraylib-0.2.5] startup
[19.664s] [ext: omni.kit.asset_converter-2.1.11] startup
2024-09-13 22:05:41 [19,907ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 0 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,909ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 1 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,911ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 2 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,912ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 3 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,914ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 4 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,916ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 5 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,918ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 6 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2024-09-13 22:05:41 [19,919ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 7 (NVIDIA H100 NVL): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
[19.969s] [ext: omni.usd.schema.isaac-2.1.0] startup
[19.978s] [ext: omni.usd.schema.physx-106.0.20] startup
[20.006s] [ext: omni.anim.navigation.schema-106.0.2] startup
[20.010s] [ext: omni.ujitso.processor.texture-1.0.0] startup
[20.012s] [ext: omni.volume-0.5.0] startup
[20.015s] [ext: omni.ujitso.client-0.0.0] startup
[20.016s] [ext: omni.kit.clipboard-1.0.3] startup
[20.017s] [ext: omni.kit.collaboration.telemetry-1.0.0] startup
[20.018s] [ext: omni.kit.widget.path_field-2.0.9] startup
[20.023s] [ext: omni.kit.widget.options_menu-1.1.4] startup
[20.034s] [ext: omni.kit.collaboration.channel_manager-1.0.11] startup
[20.039s] [ext: omni.kit.widget.browser_bar-2.0.10] startup
[20.043s] [ext: omni.kit.widget.options_button-1.0.2] startup
[20.045s] [ext: omni.kit.widget.context_menu-1.2.1] startup
[20.051s] [ext: omni.kit.notification_manager-1.0.8] startup
[20.060s] [ext: omni.inspect-1.0.1] startup
[20.062s] [ext: omni.kvdb-106.0.20] startup
[20.065s] [ext: omni.physx.foundation-106.0.20] startup
[20.065s] [ext: omni.localcache-106.0.20] startup
[20.067s] [ext: omni.convexdecomposition-106.0.20] startup
[20.072s] [ext: omni.kit.helper.file_utils-0.1.8] startup
[20.077s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[20.079s] [ext: omni.usdphysics-106.0.20] startup
[20.082s] [ext: omni.kit.widget.filebrowser-2.10.48] startup
[20.111s] [ext: omni.kit.search_core-1.0.5] startup
[20.114s] [ext: omni.physx.cooking-106.0.20] startup
[20.126s] [ext: omni.ui.scene-1.9.3] startup
[20.133s] [ext: omni.kit.widget.versioning-1.4.6] startup
[20.143s] [ext: omni.kit.widget.search_delegate-1.0.4] startup
[20.148s] [ext: omni.physx-106.0.20] startup
[20.167s] [ext: omni.kit.window.filepicker-2.10.34] startup
[20.243s] [ext: omni.kit.usd.layers-2.1.31] startup
[20.275s] [ext: omni.kit.menu.utils-1.5.27] startup
[20.317s] [ext: omni.mdl-52.0.1] startup
[20.463s] [ext: omni.kit.window.file_importer-1.1.11] startup
[20.468s] [ext: omni.kit.context_menu-1.8.0] startup
[20.478s] [ext: omni.isaac.version-1.1.0] startup
[20.480s] [ext: omni.physics.tensors-106.0.20] startup
[20.733s] [ext: omni.kit.material.library-1.4.4] startup
[20.758s] [ext: omni.isaac.nucleus-0.3.0] startup
[20.762s] [ext: omni.graph.core-2.170.3] startup
[20.766s] [ext: omni.physx.tensors-106.0.20] startup
[20.793s] [ext: omni.warp.core-1.2.1] startup
[20.990s] [ext: omni.physx.stageupdate-106.0.20] startup
[20.996s] [ext: omni.kit.widget.searchable_combobox-1.0.6] startup
[21.001s] [ext: omni.kit.widget.prompt-1.0.7] startup
[21.005s] [ext: omni.isaac.core-3.18.1] startup
[23.377s] [ext: omni.hydra.rtx-0.2.0] startup
[23.392s] [ext: omni.graph.image.core-0.3.2] startup
[23.397s] [ext: omni.kit.widget.settings-1.1.1] startup
[23.408s] [ext: omni.kit.widget.live_session_management.ui-1.0.1] startup
[23.421s] [ext: omni.graph.tools-1.78.0] startup
[23.546s] [ext: omni.kit.window.preferences-1.5.3] startup
[23.568s] [ext: omni.kit.collaboration.presence_layer-1.0.8] startup
[23.578s] [ext: omni.sensors.tiled-0.0.4] startup
[23.581s] [ext: omni.kit.widget.live_session_management-1.2.18] startup
[23.593s] [ext: omni.hydra.engine.stats-1.0.2] startup
[23.597s] [ext: omni.kit.raycast.query-1.0.5] startup
[23.605s] [ext: omni.kit.viewport.registry-104.0.6] startup
[23.606s] [ext: omni.kit.viewport.legacy_gizmos-1.0.15] startup
[23.610s] [ext: omni.kit.hydra_texture-1.2.6] startup
[23.615s] [ext: omni.kit.window.content_browser_registry-0.0.6] startup
[23.618s] [ext: omni.kit.widget.searchfield-1.1.6] startup
[23.623s] [ext: omni.kit.widget.highlight_label-1.0.2] startup
[23.626s] [ext: omni.kit.widget.viewport-106.0.3] startup
[23.647s] [ext: omni.kit.stage_template.core-1.1.21] startup
[23.649s] [ext: omni.kit.primitive.mesh-1.0.16] startup
[23.666s] [ext: omni.kit.window.file_exporter-1.0.29] startup
[23.669s] [ext: omni.kit.stage_templates-1.2.3] startup
[23.678s] [ext: omni.kit.widget.filter-1.1.4] startup
[23.680s] [ext: omni.kit.hotkeys.core-1.3.3] startup
[23.682s] [ext: omni.kit.window.file-1.3.52] startup
[23.696s] [ext: omni.kit.window.drop_support-1.0.2] startup
[23.698s] [ext: omni.kit.widget.stage-2.10.26] startup
[23.744s] [ext: omni.kit.window.content_browser-2.9.14] startup
[23.773s] [ext: omni.kit.window.property-1.11.1] startup
[23.784s] [ext: omni.kit.viewport.window-106.0.8] startup
[23.835s] [ext: omni.kit.widget.graph-1.12.8] startup
[23.873s] [ext: omni.graph-1.135.0] startup
[24.008s] [ext: omni.kit.property.usd-3.21.28] startup
[24.076s] [ext: omni.kit.viewport.utility-1.0.17] startup
[24.084s] [ext: omni.kit.window.extensions-1.4.9] startup
[24.121s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[24.123s] [ext: omni.kit.widget.material_preview-1.0.16] startup
[24.131s] [ext: omni.ui_query-1.1.2] startup
[24.134s] [ext: omni.graph.action_core-1.1.4] startup
[24.142s] [ext: omni.graph.ui-1.70.0] startup
[24.204s] [ext: omni.kit.ui_test-1.2.18] startup
[24.214s] [ext: omni.kit.manipulator.viewport-107.0.0] startup
[24.218s] [ext: omni.graph.action_nodes-1.23.0] startup
[24.228s] [ext: omni.kit.window.cursor-1.1.2] startup
[24.233s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[24.251s] [ext: omni.graph.action-1.102.1] startup
[24.254s] [ext: omni.kit.viewport.menubar.core-106.0.2] startup
[24.403s] [ext: omni.graph.image.nodes-1.0.2] startup
[24.405s] [ext: omni.isaac.ui-0.16.0] startup
[24.438s] [ext: omni.syntheticdata-0.6.7] startup
[24.532s] [ext: omni.isaac.debug_draw-1.1.0] startup
[24.542s] [ext: omni.kit.viewport.actions-106.0.3] startup
[24.577s] [ext: omni.isaac.range_sensor-3.1.1] startup
[24.665s] [ext: omni.kit.viewport.menubar.display-106.0.2] startup
[24.675s] [ext: omni.debugdraw-0.1.3] startup
[24.681s] [ext: omni.kit.widget.toolbar-1.6.2] startup
[24.704s] [ext: omni.kit.manipulator.transform-104.7.5] startup
[24.715s] [ext: omni.kit.widget.imageview-1.0.3] startup
[24.717s] [ext: omni.usdphysics.ui-106.0.20] startup
[24.747s] [ext: omni.physx.commands-106.0.20] startup
[24.769s] [ext: omni.kit.manipulator.tool.snap-1.4.5] startup
[24.794s] [ext: omni.kit.window.quicksearch-2.4.4] startup
[24.806s] [ext: omni.physx.ui-106.0.20] startup
[24.892s] [ext: omni.kit.property.material-1.9.4] startup
[24.910s] [ext: omni.physx.demos-106.0.20] startup
[25.036s] [ext: omni.kit.property.physx-106.0.20] startup
[25.166s] [ext: omni.kit.manipulator.selector-1.1.1] startup
[25.174s] [ext: omni.physx.vehicle-106.0.20] startup
[25.252s] [ext: omni.kit.viewport.manipulator.transform-106.0.1] startup
[25.259s] [ext: omni.kit.graph.delegate.default-1.2.2] startup
[25.263s] [ext: omni.kit.manipulator.prim.core-107.0.3] startup
[25.292s] [ext: omni.fabric.commands-1.1.4] startup
[25.309s] [ext: omni.kit.graph.editor.core-1.5.3] startup
[25.324s] [ext: omni.kit.manipulator.prim.fabric-106.0.1] startup
[25.329s] [ext: omni.kit.manipulator.prim.usd-106.0.1] startup
[25.335s] [ext: omni.kit.graph.usd.commands-1.3.1] startup
[25.340s] [ext: omni.physx.camera-106.0.20] startup
[25.354s] [ext: omni.physx.cct-106.0.20] startup
[25.374s] [ext: omni.kit.manipulator.selection-104.0.9] startup
[25.385s] [ext: omni.kit.manipulator.prim-106.0.0] startup
[25.387s] [ext: omni.physx.graph-106.0.20] startup
[25.426s] [ext: omni.sensors.nv.common-1.2.2-isaac] startup
[25.444s] [ext: omni.kit.window.toolbar-1.6.1] startup
[25.457s] [ext: omni.kit.widget.layers-1.7.9] startup
[25.532s] [ext: omni.command.usd-1.0.3] startup
[25.538s] [ext: omni.kit.numpy.common-0.1.2] startup
[25.543s] [ext: omni.sensors.nv.wpm-1.2.1-isaac] startup
[25.548s] [ext: omni.physx.supportui-106.0.20] startup
[25.594s] [ext: omni.physx.telemetry-106.0.20] startup
[25.604s] [ext: omni.graph.nodes-1.143.0] startup
Warning: Possible version incompatibility. Attempting to load omni::fabric::IPath with version v0.2 against v0.1.
[25.615s] [ext: omni.graph.scriptnode-1.19.1] startup
[25.621s] [ext: omni.graph.ui_nodes-1.25.1] startup
Warning: Possible version incompatibility. Attempting to load omni::fabric::IPath with version v0.2 against v0.1.
[25.630s] [ext: omni.warp-1.2.1] startup
[25.674s] [ext: omni.physx.bundle-106.0.20] startup
[25.676s] [ext: omni.kit.window.material_graph-1.8.15] startup
[26.166s] [ext: omni.graph.bundle.action-2.0.4] startup
[26.171s] [ext: omni.sensors.nv.materials-1.2.1-isaac] startup
[26.180s] [ext: omni.replicator.core-1.11.14] startup
2024-09-13 22:05:48 [26,525ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
Warp 1.2.1 initialized:
   CUDA Toolkit 11.8, Driver 12.4
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:1"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:2"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:3"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:4"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:5"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:6"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
     "cuda:7"   : "NVIDIA H100 NVL" (93 GiB, sm_90, mempool enabled)
   CUDA peer access:
     Supported fully (all-directional)
   Kernel cache:
     /root/.cache/warp/1.2.1
[27.992s] [ext: omni.isaac.dynamic_control-1.3.8] startup
[28.040s] [ext: omni.sensors.nv.radar-1.2.1-isaac] startup
[28.046s] [ext: omni.sensors.nv.lidar-1.2.2-isaac] startup
[28.051s] [ext: omni.isaac.core_nodes-1.16.1] startup
[28.097s] [ext: omni.isaac.surface_gripper-1.0.1] startup
[28.134s] [ext: omni.isaac.sensor-12.7.1] startup
[28.538s] [ext: omni.isaac.lula-3.0.1] startup
[28.597s] [ext: omni.isaac.motion_generation-7.1.0] startup
[28.882s] [ext: omni.isaac.manipulators-2.1.0] startup
[28.956s] [ext: omni.isaac.quadruped-1.4.5] startup
[29.125s] [ext: omni.isaac.franka-0.4.1] startup
[29.143s] [ext: omni.kit.graph.delegate.modern-1.10.6] startup
[29.186s] [ext: omni.isaac.wheeled_robots-2.3.3] startup
[29.215s] [ext: omni.isaac.universal_robots-0.3.5] startup
[29.230s] [ext: omni.isaac.cortex-0.3.8] startup
[29.242s] [ext: omni.kit.graph.widget.variables-2.1.0] startup
[29.264s] [ext: omni.isaac.cortex.sample_behaviors-1.0.5] startup
[29.270s] [ext: omni.graph.window.core-1.109.0] startup
[29.432s] [ext: omni.isaac.examples-2.2.1] startup
[29.990s] [ext: omni.importer.mjcf-1.1.1] startup
[30.077s] [ext: omni.graph.window.generic-1.24.0] startup
[30.134s] [ext: omni.importer.onshape-0.7.1] startup
[30.160s] [ext: omni.isaac.manipulators.ui-1.2.1] startup
[30.172s] [ext: omni.importer.urdf-1.14.1] startup
[30.268s] [ext: omni.kit.streamsdk.plugins-3.2.1] startup
[30.276s] [ext: omni.isaac.robot_description_editor-2.5.0] startup
[30.368s] [ext: omni.isaac.examples_nodes-0.1.3] startup
[30.382s] [ext: omni.isaac.cloner-0.8.1] startup
[30.402s] [ext: omni.isaac.extension_templates-1.7.0] startup
[30.413s] [ext: omni.isaac.window.about-1.1.0] startup
[30.433s] [ext: omni.isaac.gym-0.11.3] startup
[30.437s] [ext: omni.isaac.import_wizard-0.5.2] startup
[30.463s] [ext: omni.isaac.kit-1.13.0] startup
[30.478s] [ext: omni.anim.curve.core-1.1.13] startup
[30.532s] [ext: omni.isaac.block_world-1.0.0] startup
[30.547s] [ext: omni.kit.preferences.animation-1.1.7] startup
[30.566s] [ext: omni.anim.curve.ui-1.3.16] startup
[30.586s] [ext: omni.isaac.occupancy_map-1.0.2] startup
[30.614s] [ext: omni.anim.shared.core-106.0.0] startup
[30.627s] [ext: omni.isaac.merge_mesh-1.0.2] startup
[30.647s] [ext: omni.anim.curve.bundle-1.2.3] startup
[30.647s] [ext: omni.isaac.occupancy_map.ui-1.1.0] startup
[30.664s] [ext: omni.kit.stage.copypaste-1.2.2] startup
[30.695s] [ext: omni.kit.widget.stage_icons-1.0.4] startup
[30.704s] [ext: omni.kit.widget.zoombar-1.0.5] startup
[30.718s] [ext: omni.kit.usd.collect-2.2.21] startup
[30.742s] [ext: omni.kit.window.stage-2.5.10] startup
[30.757s] [ext: omni.kit.browser.core-2.3.11] startup
[30.792s] [ext: omni.kit.widgets.custom-1.0.8] startup
[30.857s] [ext: omni.kit.tool.collect-2.2.14] startup
[30.879s] [ext: omni.kit.menu.stage-1.2.5] startup
[30.890s] [ext: omni.kit.browser.folder.core-1.9.12] startup
[30.927s] [ext: omni.kit.variant.editor-106.0.0] startup
[31.029s] [ext: omni.kit.core.collection-0.1.7] startup
[31.040s] [ext: omni.isaac.physics_inspector-0.1.4] startup
[31.050s] [ext: omni.isaac.asset_browser-0.5.0] startup
[31.073s] [ext: omni.kit.variant.presenter-105.1.2] startup
[31.095s] [ext: omni.isaac.scene_blox-0.1.2] startup
[31.098s] [ext: omni.isaac.surface_gripper.ui-1.0.0] startup
[31.106s] [ext: omni.isaac.synthetic_recorder-1.8.0] startup
[31.128s] [ext: omni.isaac.throttling-1.1.0] startup
[31.136s] [ext: omni.isaac.utils-1.0.1] startup
[31.148s] [ext: omni.isaac.gain_tuner-1.1.2] startup
[31.166s] [ext: omni.kit.window.usd_paths-1.0.6] startup
[31.182s] [ext: omni.kit.actions.window-1.1.1] startup
[31.205s] [ext: omni.replicator.isaac-1.15.0] startup
[31.323s] [ext: omni.kit.selection-0.1.4] startup
[31.335s] [ext: omni.kit.browser.asset-1.3.9] startup
[31.354s] [ext: omni.kit.hotkeys.window-1.4.5] startup
[31.437s] [ext: omni.kit.menu.aov-1.1.3] startup
[31.453s] [ext: omni.kit.menu.edit-1.1.24] startup
[31.486s] [ext: omni.kit.menu.file-1.1.10] startup
[31.502s] [ext: omni.kit.property.audio-1.0.11] startup
[31.512s] [ext: omni.kit.property.camera-1.0.6] startup
[31.522s] [ext: omni.hydra.scene_api-0.1.2] startup
[31.531s] [ext: omni.kit.property.geometry-1.3.0] startup
[31.548s] [ext: omni.kit.property.light-1.0.8] startup
[31.558s] [ext: omni.kit.property.render-1.1.1] startup
[31.566s] [ext: omni.kit.property.transform-1.5.1] startup
[31.602s] [ext: omni.kit.mesh.raycast-105.4.0] startup
[31.608s] [ext: omni.kit.window.title-1.1.3] startup
2024-09-13 22:05:53 [31,584ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:53 [31,584ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-09-13 22:05:53 [31,589ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-09-13 22:05:53 [31,589ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[31.620s] [ext: omni.isaac.menu-0.5.0] startup
[31.632s] [ext: omni.kit.property.bundle-1.2.11] startup
[31.648s] [ext: omni.kit.property.collection-0.1.17] startup
[31.655s] [ext: omni.kit.menu.create-1.0.13] startup
[31.664s] [ext: omni.kit.property.isaac-0.2.3] startup
[31.675s] [ext: omni.kit.property.layer-1.1.6] startup
[31.692s] [ext: omni.kit.stagerecorder.core-105.0.5] startup
[31.702s] [ext: omni.mdl.usd_converter-1.0.19] startup
[31.733s] [ext: omni.kit.scripting-106.0.1] startup
[31.767s] [ext: omni.kit.search.files-1.0.4] startup
[31.774s] [ext: semantics.schema.property-1.0.3] startup
[31.783s] [ext: omni.kit.stagerecorder.ui-105.0.6] startup
[31.803s] [ext: omni.kit.stage.mdl_converter-1.0.5] startup
[31.811s] [ext: omni.kit.stage_column.payload-2.0.0] startup
[31.820s] [ext: omni.isaac.physics_utilities-0.1.4] startup
[31.830s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[31.848s] [ext: omni.kit.stagerecorder.bundle-105.0.2] startup
[31.849s] [ext: omni.isaac.range_sensor.examples-1.0.0] startup
[31.881s] [ext: omni.graph.window.action-1.26.0] startup
[31.913s] [ext: omni.kit.viewport.menubar.render-106.1.3] startup
[31.939s] [ext: omni.kit.viewport.menubar.settings-106.0.1] startup
[31.970s] [ext: omni.kit.viewport.menubar.camera-105.1.8] startup
[32.005s] [ext: omni.kit.manipulator.camera-105.0.5] startup
[32.035s] [ext: omni.rtx.window.settings-0.6.16] startup
[32.056s] [ext: omni.kit.widget.live-2.1.6] startup
[32.066s] [ext: omni.kit.widget.calendar-1.0.8] startup
[32.079s] [ext: omni.kit.viewport.bundle-104.0.1] startup
[32.080s] [ext: omni.kit.widget.collection-0.1.18] startup
[32.101s] [ext: omni.kit.viewport.menubar.lighting-106.0.2] startup
[32.129s] [ext: omni.kit.viewport.rtx-104.0.1] startup
[32.129s] [ext: omni.kit.viewport_widgets_manager-1.0.8] startup
[32.140s] [ext: omni.kit.widget.cache_indicator-2.0.8] startup
[32.386s] [ext: omni.kit.widget.extended_searchfield-1.0.28] startup
2024-09-13 22:05:54 [32,359ms] [Warning] [omni.kit.widget.cache_indicator.cache_state_menu] Unable to detect Omniverse Cache Server. Consider installing it for better IO performance.
[32.436s] [ext: omni.kit.widget.timeline-105.0.1] startup
[32.460s] [ext: omni.kit.window.collection-0.1.22] startup
[32.469s] [ext: omni.kit.browser.material-1.5.2] startup
[32.493s] [ext: omni.kit.window.commands-0.2.5] startup
[32.506s] [ext: omni.kit.window.console-0.2.12] startup
[32.529s] [ext: omni.kit.thumbnails.usd-1.0.9] startup
[32.867s] [ext: omni.kit.window.privacy-0.1.0] startup
[32.873s] [ext: omni.scene.optimizer.core-106.0.15] startup
[32.967s] [ext: omni.kit.tool.remove_unused.core-0.1.2] startup
[32.973s] [ext: omni.kit.window.material-1.5.7] startup
[33.014s] [ext: omni.kit.window.script_editor-1.7.6] startup
[33.026s] [ext: omni.isaac.wheeled_robots.ui-1.1.1] startup
[33.034s] [ext: omni.kit.tool.asset_exporter-1.3.3] startup
[33.049s] [ext: omni.isaac.range_sensor.ui-1.0.1] startup
[33.057s] [ext: omni.kit.converter.common-500.0.8] startup
[33.128s] [ext: omni.kit.window.status_bar-0.1.6] startup
[33.135s] [ext: omni.kit.tool.remove_unused.controller-0.1.3] startup
[33.143s] [ext: omni.replicator.replicator_yaml-2.0.5] startup
[33.165s] [ext: omni.rtx.settings.core-0.6.0] startup
[33.200s] [ext: omni.usd.metrics.assembler-106.0.1] startup
[33.223s] [ext: omni.kit.converter.dgn_core-500.0.12] startup
[33.306s] [ext: omni.kit.ui.actions-1.0.1] startup
[33.315s] [ext: omni.kit.tool.asset_importer-2.5.5] startup
[33.317s] [ext: omni.kit.converter.hoops_core-500.0.12] startup
[33.330s] [ext: omni.simready.explorer-1.0.26] startup
[33.391s] [ext: omni.kit.converter.jt_core-500.0.12] startup
[33.420s] [ext: omni.usd.metrics.assembler.physics-106.0.2] startup
[33.428s] [ext: omni.usd.metrics.assembler.ui-106.0.2] startup
[33.450s] [ext: omni.kit.menu.common-1.1.5] startup
[33.459s] [ext: omni.kit.converter.dgn-500.0.4] startup
[33.482s] [ext: omni.kit.converter.hoops-500.0.5] startup
[33.491s] [ext: omni.kit.renderer.cuda_interop-1.0.1] startup
[33.497s] [ext: omni.kit.converter.jt-500.0.4] startup
[33.517s] [ext: omni.kit.quicklayout-1.0.7] startup
[33.524s] [ext: semantics.schema.editor-0.3.6] startup
[33.534s] [ext: omni.kit.usda_edit-1.1.11] startup
[33.579s] [ext: omni.isaac.robot_assembler-1.5.0] startup
[33.593s] [ext: omni.isaac.camera_inspector-0.2.2] startup
[33.605s] [ext: omni.kit.livestream.core-3.2.0] startup
[33.611s] [ext: omni.kit.converter.cad-201.1.0-rc.4] startup
[33.612s] [ext: omni.kit.window.stats-0.1.6] startup
[33.617s] [ext: omni.isaac.app.setup-0.13.2] startup
[33.628s] Isaac Sim Headless Native Version: 4.1.0-rc.7
[33.643s] [ext: omni.isaac.assets_check-0.3.4] startup
[33.660s] [ext: omni.kit.profiler.window-2.2.1] startup
2024-09-13 22:05:55 [33,702ms] [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
[33.746s] [ext: omni.isaac.sim.base-4.1.0] startup
[33.752s] [ext: omni.kit.livestream.native-4.1.0] startup

Failed to open [/var/run/utmp]
Active user not found. Using default user [kiosk]Streaming server started.
[39.193s] [ext: omni.isaac.sim.headless.native-4.1.0] startup
[39.489s] [ext: omni.isaac.ros2_bridge-2.31.4] startup
2024-09-13 22:06:02 [40,118ms] [Warning] [omni.isaac.ros2_bridge.scripts.extension] ROS_DISTRO env var not found, Please source ROS2 Foxy, or Humble, before enabling this extension
[40.148s] Using backup internal ROS2 humble distro
Checking to see if RMW can be loaded:
failed to get symbol 'rmw_init_options_init' due to Environment variable 'AMENT_PREFIX_PATH' is not set or empty, at /workspace/humble_ws/src/rmw_implementation/src/functions.cpp:171, at /workspace/humble_ws/src/rcl/rcl/src/rcl/init_options.c:75
RMW was not loaded

[40.164s] To use the internal libraries included with the extension please set the following environment variables to use with FastDDS (default) or CycloneDDS (ROS2 Humble only): 
RMW_IMPLEMENTATION=rmw_fastrtps_cpp
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/isaac-sim/exts/omni.isaac.ros2_bridge/humble/lib

OR 

RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/isaac-sim/exts/omni.isaac.ros2_bridge/humble/lib
Before starting Isaac Sim
2024-09-13 22:06:02 [40,135ms] [Error] [omni.isaac.ros2_bridge.scripts.extension] ROS2 Bridge startup failed
[75.256s] [ext: omni.isaac.ros2_bridge-2.31.4] shutdown
2024-09-13 22:06:37 [75,385ms] [Warning] [omni.ext._impl._internal] omni.isaac.ros2_bridge-2.31.4 -> <class 'omni.isaac.ros2_bridge.scripts.og_shortcuts.og_shortcut_menu.Extension'>: extension object is still alive, something holds a reference on it. References: ["[0]:type: <class 'method'>, id: 140425045745536", "[1]:type: <class 'method'>, id: 140425092213376", "[2]:type: <class 'method'>, id: 140425045752960", "[3]:type: <class 'method'>, id: 140431943400448", "[4]:type: <class 'method'>, id: 140424885844928", '[*] ...']
[96.474s] app ready
2024-09-13 22:06:58 [96,700ms] [Warning] [omni.kit.imgui_renderer.plugin] _createExtendCursor: No windowing.
2024-09-13 22:06:58 [96,700ms] [Warning] [omni.kit.imgui_renderer.plugin] _createExtendCursor: No windowing.
[96.909s] Isaac Sim Headless Native App is loaded.

When I try to stream to a client computer, a window very briefly pops up and disappears, then on the server, I see

main: thread_init: already added for thread
main: thread_init: already added for thread

When using the -v flag for verbose output, I get a few more lines of error output:

main: thread_init: already added for thread
main: thread_init: already added for thread
2024-09-13 22:28:39 [121,502ms] [Info] [carb.livestream.plugin] Stream server: connected stream 0x7fbb38007a40 on connection 0x7fbaa0012260
2024-09-13 22:28:40 [122,444ms] [Info] [carb.livestream.plugin] Client (nil) disconnected.
2024-09-13 22:28:40 [122,462ms] [Info] [carb.livestream.plugin] Client (nil) disconnected.
2024-09-13 22:28:40 [122,636ms] [Info] [carb.livestream.plugin] Stream Server: stream0 0x7fbaa0012260 (type 1) stopped
2024-09-13 22:28:40 [122,636ms] [Info] [carb.livestream.plugin] Stream Server: stream1 0x7fbaa0007440 (type 4) stopped
2024-09-13 22:28:40 [122,636ms] [Info] [carb.livestream.plugin] Client (nil) disconnected.

On the client, I'm using Omniverse Streaming Client v103.1.1.

Are there any workarounds to this issue?

[jtigue-bdai]

@alberthli, it looks like you are using the fix that came with this issue so I am surprised its not working.

Can you try: settingenable_extension("omni.kit.livestream.native-4.1.0") to enable_extension("omni.kit.livestream.native") in your source/extensions/omni.isaac.lab/omni/isaac/lab/app/app_launcher.py file?

Let me know if that works. If that doesn't work please open another ticket.

[alberthli]

@jtigue-bdai, thanks for the quick response. Sorry if I'm missing something obvious (I'm new to Isaac Lab), but since I'm starting up the server using the script in the Isaac Docker container, I'm not sure I have or am using the app_launcher.py file. I grepd the phrase

omni.kit.livestream.native

in the isaac-sim directory of the container, and it seems like the version for this is just set in a bunch of .kit files. In the file apps/omni.isaac.sim.headless.native.kit, I changed

"omni.kit.livestream.native" = {version = "4.1.0", exact = true}

to

"omni.kit.livestream.native" = {version = "*"}

and ran into the same issue. If I'm doing something wrong, please let me know. Otherwise, I'll open a new issue.

[josyulavt]

@jtigue-bdai, thanks for the quick response. Sorry if I'm missing something obvious (I'm new to Isaac Lab), but since I'm starting up the server using the script in the Isaac Docker container, I'm not sure I have or am using the app_launcher.py file. I grepd the phrase

omni.kit.livestream.native

in the isaac-sim directory of the container, and it seems like the version for this is just set in a bunch of .kit files. In the file apps/omni.isaac.sim.headless.native.kit, I changed

"omni.kit.livestream.native" = {version = "4.1.0", exact = true}

to

"omni.kit.livestream.native" = {version = "*"}

and ran into the same issue. If I'm doing something wrong, please let me know. Otherwise, I'll open a new issue.

I'm also running into the exact same issue as this user mentioned. Can someone help? thanks

[jtigue-bdai]

@alberthli and @josyulavt are you both using NVidia H100 GPUs? If so they cannot run Isaac Sim. requirements

[josyulavt]

@alberthli and @josyulavt are you both using NVidia H100 GPUs? If so they cannot run Isaac Sim. requirements

hi @jtigue-bdai my case may be a little different, this is my setup:

• Issac Sim docker container
• cuRobo installed in the docker
• Host omniverse streamer

when I run runheadless.native I can see issacsim running on the omniverse streaming client.

however when I run the curobo example

/isaac-sim/python.sh examples/isaac_sim/collision_checker.py --headless_mode native

this error shows up this is the code for starting the simulation

try:
    # Third Party
    import isaacsim
except ImportError:
    pass

# Third Party
import torch

a = torch.zeros(4, device="cuda:0")

# Standard Library

# Standard Library
import argparse

# Third Party
from omni.isaac.kit import SimulationApp

parser = argparse.ArgumentParser()
parser.add_argument(
    "--nvblox", action="store_true", help="When True, enables headless mode", default=False
)

parser.add_argument(
    "--headless_mode",
    type=str,
    default=None,
    help="To run headless, use one of [native, websocket], webrtc might not work.",
)
args = parser.parse_args()

simulation_app = SimulationApp(
    {
        "headless": args.headless_mode is not None,
        "width": "1920",
        "height": "1080",
    }
)

full log

Starting kit application with the following args:  ['/isaac-sim/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/isaac-sim/apps/omni.isaac.sim.python.kit', '--/app/tokens/exe-path=/isaac-sim/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True', '--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1920', '--/app/renderer/resolution/height=1080', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/isaac-sim/exts', '--ext-folder', '/isaac-sim/apps', '--/physics/cudaDevice=0', '--portable', '--no-window', '--/app/window/hideUi=1', '--allow-root']
Passing the following args to the base kit application:  ['--headless_mode', 'native']
[Info] [carb] Logging to file: /isaac-sim/kit/logs/Kit/Isaac-Sim/4.2/kit_20241018_164209.log
2024-10-18 16:42:09 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
[0.091s] [ext: omni.blobkey-1.1.2] startup
[0.091s] [ext: omni.kit.async_engine-0.0.1] startup
[0.589s] [ext: omni.stats-1.0.1] startup
[0.590s] [ext: omni.datastore-0.0.0] startup
[0.594s] [ext: omni.client-1.2.0] startup
[0.612s] [ext: omni.ujitso.default-1.0.0] startup
[0.613s] [ext: omni.gpu_foundation.shadercache.vulkan-1.0.0] startup
[0.615s] [ext: omni.hsscclient-1.1.1] startup
[0.615s] [ext: omni.assets.plugins-0.0.0] startup
[0.616s] [ext: omni.gpu_foundation-0.0.0] startup
[0.622s] [ext: carb.windowing.plugins-1.0.0] startup
2024-10-18 16:42:09 [556ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:09 [556ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[0.623s] [ext: omni.kit.renderer.init-0.0.0] startup
2024-10-18 16:42:09 [579ms] [Warning] [omni.platforminfo.plugin] failed to open the default display.  Can't verify X Server version.

|---------------------------------------------------------------------------------------------|
| Driver Version: 550.107.02    | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 4080 SUPER    | Yes: 0 |     | 16376   MB | 10de      | 0          |
|     |                                  |        |     |            | 2702      | e510b4eb.. |
|     |                                  |        |     |            | 1         |            |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 6.8.0-47-generic
| Processor: AMD Ryzen 9 7950X3D 16-Core Processor | Cores: 16 | Logical: 32
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 95681 | Free Memory: 88572
| Total Page/Swap (MB): 8191 | Free Page/Swap: 8191
|---------------------------------------------------------------------------------------------|
2024-10-18 16:42:09 [815ms] [Warning] [gpu.foundation.plugin] IOMMU is enabled.
[1.019s] [ext: omni.kit.pipapi-0.0.0] startup
[1.021s] [ext: omni.kit.pip_archive-0.0.0] startup
[1.021s] [ext: omni.pip.compute-1.4.0] startup
[1.021s] [ext: omni.pip.cloud-1.1.3] startup
[1.023s] [ext: omni.isaac.core_archive-2.3.0] startup
[1.023s] [ext: omni.materialx.libs-1.0.4] startup
[1.029s] [ext: omni.isaac.ml_archive-2.0.1] startup
[1.029s] [ext: omni.kit.telemetry-0.5.0] startup
[1.060s] [ext: omni.usd.config-1.0.4] startup
[1.064s] [ext: omni.gpucompute.plugins-0.0.0] startup
[1.064s] [ext: omni.usd.libs-1.0.1] startup
[1.124s] [ext: omni.iray.libs-0.0.0] startup
[1.128s] [ext: omni.mdl.neuraylib-0.2.8] startup
[1.130s] [ext: omni.mdl-55.0.1] startup
[1.146s] [ext: omni.kit.usd.mdl-1.0.1] startup
[1.224s] [ext: omni.kit.loop-isaac-1.2.0] startup
[1.225s] [ext: omni.kit.test-1.1.0] startup
[1.253s] [ext: omni.appwindow-1.1.8] startup
2024-10-18 16:42:10 [1,187ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:10 [1,187ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[1.255s] [ext: omni.kit.renderer.core-1.0.2] startup
2024-10-18 16:42:10 [1,192ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:10 [1,192ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-10-18 16:42:10 [1,197ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:10 [1,197ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[1.265s] [ext: omni.kit.renderer.capture-0.0.0] startup
[1.266s] [ext: omni.kit.renderer.imgui-1.0.1] startup
2024-10-18 16:42:10 [1,201ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:10 [1,201ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-10-18 16:42:10 [1,202ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:10 [1,202ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
2024-10-18 16:42:10 [1,203ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2024-10-18 16:42:10 [1,203ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.4]) (impl: carb.windowing-glfw.plugin)
[1.316s] [ext: omni.ui-2.25.22] startup
[1.324s] [ext: omni.kit.mainwindow-1.0.3] startup
[1.324s] [ext: carb.audio-0.1.0] startup
[1.328s] [ext: omni.uiaudio-1.0.0] startup
[1.329s] [ext: omni.kit.uiapp-0.0.0] startup
[1.329s] [ext: omni.usd.schema.isaac-2.1.0] startup
[1.335s] [ext: omni.usd.schema.audio-0.0.0] startup
[1.337s] [ext: omni.usd.schema.anim-0.0.0] startup
[1.351s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[1.353s] [ext: omni.usd.schema.semantics-0.0.0] startup
[1.355s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[1.359s] [ext: omni.anim.navigation.schema-106.1.0] startup
[1.361s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[1.363s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[1.366s] [ext: omni.anim.graph.schema-106.1.0] startup
[1.370s] [ext: omni.usd.schema.physx-106.1.9] startup
[1.392s] [ext: omni.usd.schema.forcefield-106.1.9] startup
[1.395s] [ext: omni.graph.exec-0.9.4] startup
[1.396s] [ext: omni.kit.usd_undo-0.1.8] startup
[1.397s] [ext: omni.kit.actions.core-1.0.0] startup
[1.398s] [ext: omni.kit.exec.core-0.13.4] startup
[1.400s] [ext: omni.usd_resolver-1.0.0] startup
[1.402s] [ext: omni.kit.commands-1.4.9] startup
[1.405s] [ext: omni.activity.core-1.0.1] startup
[1.406s] [ext: omni.usd.core-1.4.1] startup
[1.408s] [ext: omni.resourcemonitor-105.0.1] startup
[1.409s] [ext: omni.kit.window.popup_dialog-2.0.24] startup
[1.412s] [ext: omni.timeline-1.0.10] startup
[1.413s] [ext: omni.kit.widget.nucleus_connector-1.1.9] startup
[1.414s] [ext: usdrt.scenegraph-7.5.0] startup
[1.443s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[1.444s] [ext: omni.hydra.usdrt_delegate-7.5.1] startup
[1.453s] [ext: omni.hydra.scene_delegate-0.3.3] startup
[1.457s] [ext: omni.usd-1.12.2] startup
[1.478s] [ext: omni.kit.menu.core-1.0.4] startup
[1.479s] [ext: omni.kit.collaboration.telemetry-1.0.0] startup
[1.479s] [ext: omni.kit.property.adapter.core-1.0.1] startup
[1.481s] [ext: omni.kit.menu.utils-1.5.27] startup
[1.486s] [ext: omni.kit.clipboard-1.0.4] startup
[1.487s] [ext: omni.kit.collaboration.channel_manager-1.0.12] startup
[1.487s] [ext: omni.kit.property.adapter.usd-1.0.1] startup
[1.488s] [ext: omni.kit.widget.options_menu-1.1.6] startup
[1.490s] [ext: omni.kit.widget.graph-1.12.15] startup
[1.493s] [ext: omni.kit.widget.searchfield-1.1.8] startup
[1.494s] [ext: omni.kit.widget.filter-1.1.4] startup
[1.495s] [ext: omni.kit.property.adapter.fabric-1.0.1] startup
[1.495s] [ext: omni.ui_query-1.1.4] startup
[1.496s] [ext: omni.kit.window.extensions-1.4.11] startup
[1.500s] [ext: omni.kit.usd.layers-2.1.36] startup
[1.505s] [ext: omni.kit.widget.context_menu-1.2.2] startup
[1.506s] [ext: omni.hydra.rtx.shadercache.vulkan-1.0.0] startup
[1.507s] [ext: omni.kit.notification_manager-1.0.9] startup
[1.508s] [ext: omni.kit.hotkeys.core-1.3.5] startup
[1.508s] [ext: omni.kit.context_menu-1.8.1] startup
[1.509s] [ext: omni.kit.widget.prompt-1.0.7] startup
[1.510s] [ext: omni.kit.helper.file_utils-0.1.8] startup
[1.511s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[1.511s] [ext: omni.kit.widget.filebrowser-2.10.51] startup
[1.514s] [ext: omni.kit.search_core-1.0.5] startup
[1.515s] [ext: omni.kit.widget.path_field-2.0.10] startup
[1.515s] [ext: omni.ui.scene-1.10.3] startup
[1.518s] [ext: omni.kit.widget.search_delegate-1.0.5] startup
[1.519s] [ext: omni.kit.widget.options_button-1.0.3] startup
[1.520s] [ext: omni.kit.widget.browser_bar-2.0.10] startup
[1.520s] [ext: omni.volume-0.5.0] startup
[1.522s] [ext: omni.kit.window.filepicker-2.10.40] startup
[1.531s] [ext: omni.ujitso.processor.texture-1.0.0] startup
[1.531s] [ext: omni.kit.window.file_exporter-1.0.30] startup
[1.533s] [ext: omni.ujitso.client-0.0.0] startup
[1.533s] [ext: omni.kit.widget.stage-2.11.2] startup
[1.540s] [ext: omni.kit.widget.live_session_management.ui-1.0.1] startup
[1.541s] [ext: omni.kit.collaboration.presence_layer-1.0.9] startup
[1.543s] [ext: omni.hydra.rtx-1.0.0] startup
[1.555s] [ext: omni.inspect-1.0.1] startup
[1.557s] [ext: omni.kit.widget.searchable_combobox-1.0.6] startup
[1.557s] [ext: omni.kit.window.file_importer-1.1.12] startup
[1.558s] [ext: omni.kit.stage_template.core-1.1.22] startup
[1.558s] [ext: omni.kit.widget.settings-1.2.2] startup
[1.560s] [ext: omni.kit.window.file-1.3.54] startup
[1.562s] [ext: omni.kit.window.drop_support-1.0.3] startup
[1.562s] [ext: omni.kit.window.content_browser_registry-0.0.6] startup
[1.562s] [ext: omni.kit.window.preferences-1.6.0] startup
[1.566s] [ext: omni.kit.window.content_browser-2.9.18] startup
[1.576s] [ext: omni.kit.widget.live_session_management-1.2.20] startup
[1.578s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[1.579s] [ext: omni.kit.hydra_texture-1.3.9] startup
[1.581s] [ext: omni.kit.viewport.legacy_gizmos-1.1.0] startup
[1.583s] [ext: omni.kit.raycast.query-1.0.5] startup
[1.587s] [ext: omni.kit.material.library-1.5.6] startup
[1.591s] [ext: omni.kit.widget.viewport-106.1.0] startup
[1.594s] [ext: omni.kit.viewport.registry-104.0.6] startup
[1.594s] [ext: omni.hydra.engine.stats-1.0.2] startup
[1.596s] [ext: omni.kit.widget.highlight_label-1.0.2] startup
[1.596s] [ext: omni.kit.viewport.window-107.0.6] startup
[1.608s] [ext: omni.kit.window.property-1.11.3] startup
[1.610s] [ext: omni.graph.core-2.179.2] startup
[1.611s] [ext: omni.graph.tools-1.79.0] startup
[1.626s] [ext: omni.kit.viewport.utility-1.0.17] startup
[1.627s] [ext: omni.kit.property.usd-4.2.8] startup
[1.633s] [ext: omni.debugdraw-0.1.3] startup
[1.635s] [ext: omni.graph-1.140.0] startup
[1.665s] [ext: omni.graph.ui-1.70.2] startup
[1.677s] [ext: omni.kvdb-106.1.9] startup
[1.680s] [ext: omni.graph.image.core-0.4.5] startup
[1.681s] [ext: omni.localcache-106.1.9] startup
[1.683s] [ext: omni.graph.image.nodes-1.1.0] startup
[1.685s] [ext: omni.usdphysics-106.1.9] startup
[1.688s] [ext: omni.physx.foundation-106.1.9] startup
[1.689s] [ext: omni.convexdecomposition-106.1.9] startup
[1.693s] [ext: omni.graph.nodes-1.146.1] startup
[1.700s] [ext: omni.physx.cooking-106.1.9] startup
[1.708s] [ext: omni.kit.primitive.mesh-1.0.17] startup
[1.710s] [ext: omni.physx-106.1.9] startup
[1.719s] [ext: omni.kit.stage_templates-1.2.5] startup
[1.721s] [ext: omni.isaac.version-1.1.0] startup
[1.722s] [ext: omni.kit.widget.material_preview-1.0.16] startup
[1.723s] [ext: omni.isaac.nucleus-0.3.1] startup
[1.724s] [ext: omni.physics.tensors-106.1.9] startup
[1.729s] [ext: omni.physx.stageupdate-106.1.9] startup
[1.730s] [ext: omni.graph.action_core-1.1.6] startup
[1.733s] [ext: omni.warp.core-1.2.1] startup
[1.804s] [ext: omni.physx.tensors-106.1.9] startup
[1.808s] [ext: omni.graph.scriptnode-1.20.1] startup
[1.809s] [ext: omni.kit.manipulator.viewport-107.0.0] startup
[1.810s] [ext: omni.graph.action_nodes-1.24.0] startup
[1.815s] [ext: omni.kit.ui_test-1.3.2] startup
[1.820s] [ext: omni.isaac.core-3.19.5] startup
[2.055s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[2.060s] [ext: omni.graph.action-1.102.1] startup
[2.062s] [ext: omni.kit.window.cursor-1.1.2] startup
[2.064s] [ext: omni.isaac.ui-0.16.0] startup
[2.072s] [ext: omni.kit.viewport.menubar.core-106.1.0] startup
[2.085s] [ext: omni.kit.graph.delegate.default-1.2.2] startup
[2.086s] [ext: omni.syntheticdata-0.6.9] startup
[2.106s] [ext: omni.kit.graph.usd.commands-1.3.1] startup
[2.109s] [ext: omni.kit.graph.editor.core-1.5.3] startup
[2.113s] [ext: omni.kit.numpy.common-0.1.2] startup
[2.115s] [ext: omni.warp-1.2.1] startup
[2.126s] [ext: omni.kit.window.material_graph-1.8.18] startup
[2.181s] [ext: omni.isaac.dynamic_control-1.3.8] startup
[2.196s] [ext: omni.replicator.core-1.11.20] startup
2024-10-18 16:42:11 [2,146ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
Warp 1.2.1 initialized:
   CUDA Toolkit 11.8, Driver 12.4
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA GeForce RTX 4080 SUPER" (16 GiB, sm_89, mempool enabled)
   Kernel cache:
     /root/.cache/warp/1.2.1
[2.306s] [ext: omni.isaac.lula-3.0.1] startup
[2.313s] [ext: omni.isaac.surface_gripper-1.0.1] startup
[2.317s] [ext: omni.isaac.core_nodes-1.16.3] startup
[2.333s] [ext: omni.isaac.motion_generation-7.1.0] startup
[2.346s] [ext: omni.isaac.manipulators-2.1.0] startup
[2.348s] [ext: omni.isaac.franka-0.4.1] startup
[2.355s] [ext: omni.kit.graph.widget.variables-2.1.0] startup
[2.362s] [ext: omni.kit.graph.delegate.modern-1.10.6] startup
[2.368s] [ext: omni.isaac.cortex-0.3.9] startup
[2.370s] [ext: omni.graph.window.core-1.113.1] startup
[2.391s] [ext: omni.isaac.wheeled_robots-2.3.3] startup
[2.406s] [ext: omni.isaac.cortex.sample_behaviors-1.0.6] startup
[2.406s] [ext: omni.graph.window.generic-1.26.0] startup
[2.418s] [ext: omni.isaac.menu-0.7.3] startup
[2.422s] [ext: omni.kit.widget.live-2.1.8] startup
[2.429s] [ext: omni.kit.actions.window-1.1.1] startup
[2.438s] [ext: omni.kit.widget.cache_indicator-2.0.10] startup
[2.494s] [ext: omni.kit.hotkeys.window-1.4.5] startup
2024-10-18 16:42:11 [2,427ms] [Warning] [omni.kit.widget.cache_indicator.cache_state_menu] Unable to detect Omniverse Cache Server. Consider installing it for better IO performance.
[2.504s] [ext: omni.kit.ui.actions-1.0.2] startup
[2.506s] [ext: omni.kit.widget.stage_icons-1.0.5] startup
[2.510s] [ext: omni.kit.menu.common-1.1.7] startup
[2.515s] [ext: omni.kit.selection-0.1.4] startup
[2.518s] [ext: omni.isaac.kit-1.13.2] startup
[2.519s] [ext: omni.isaac.block_world-1.0.0] startup
[2.526s] [ext: omni.kit.window.stage-2.5.10] startup
[2.529s] [ext: omni.isaac.debug_draw-1.1.0] startup
[2.532s] [ext: omni.kit.menu.edit-1.1.24] startup
[2.536s] [ext: omni.kit.menu.file-1.1.14] startup
[2.537s] [ext: omni.kit.profiler.window-2.2.3] startup
[2.543s] [ext: omni.kit.menu.stage-1.2.5] startup
[2.545s] [ext: omni.isaac.occupancy_map-1.0.3] startup
[2.549s] [ext: omni.importer.mjcf-1.1.1] startup
[2.554s] [ext: omni.graph.window.action-1.28.0] startup
[2.559s] [ext: omni.kit.property.camera-1.0.8] startup
[2.561s] [ext: omni.kit.property.light-1.0.10] startup
[2.562s] [ext: omni.kit.property.geometry-1.3.1] startup
[2.565s] [ext: omni.hydra.scene_api-0.1.2] startup
[2.568s] [ext: omni.kit.property.audio-1.0.14] startup
[2.570s] [ext: omni.kit.property.render-1.1.2] startup
[2.571s] [ext: omni.kit.property.transform-1.5.9] startup
[2.574s] [ext: omni.kit.property.material-1.10.8] startup
[2.578s] [ext: omni.kit.widget.layers-1.8.0] startup
[2.586s] [ext: omni.kit.property.isaac-0.2.3] startup
[2.587s] [ext: omni.kit.property.bundle-1.3.1] startup
[2.588s] [ext: omni.sensors.nv.common-1.2.2-isaac-1] startup
[2.596s] [ext: omni.kit.property.layer-1.1.8] startup
[2.598s] [ext: omni.kit.stage_column.payload-2.0.0] startup
[2.599s] [ext: omni.isaac.scene_blox-0.1.2] startup
[2.600s] [ext: omni.sensors.nv.materials-1.2.1-isaac-1] startup
[2.602s] [ext: omni.sensors.nv.wpm-1.2.1-isaac-1] startup
[2.602s] [ext: omni.kit.viewport.actions-106.0.2] startup
[2.605s] [ext: omni.sensors.nv.lidar-1.2.2-isaac-1] startup
[2.608s] [ext: omni.sensors.nv.radar-1.2.1-isaac-1] startup
[2.613s] [ext: omni.kit.viewport.menubar.display-107.0.2] startup
[2.614s] [ext: omni.kit.manipulator.transform-104.7.4] startup
[2.616s] [ext: omni.kit.widget.toolbar-1.7.2] startup
[2.620s] [ext: omni.sensors.tiled-0.0.6] startup
[2.622s] [ext: omni.sensors.nv.ids-1.1.0-isaac-1] startup
[2.625s] [ext: omni.isaac.range_sensor-3.1.2] startup
[2.633s] [ext: omni.usdphysics.ui-106.1.9] startup
[2.648s] [ext: omni.physx.commands-106.1.9] startup
[2.651s] [ext: omni.kit.manipulator.tool.snap-1.5.11] startup
[2.655s] [ext: omni.isaac.sensor-12.9.1] startup
[2.701s] [ext: omni.kit.manipulator.selector-1.1.1] startup
[2.704s] [ext: omni.physx.ui-106.1.9] startup
[2.731s] [ext: omni.isaac.quadruped-2.0.1] startup
[2.737s] [ext: omni.kit.viewport.manipulator.transform-107.0.2] startup
[2.739s] [ext: omni.physx.demos-106.1.9] startup
[2.752s] [ext: omni.kit.property.physx-106.1.9] startup
[2.803s] [ext: omni.kit.widget.calendar-1.0.8] startup
[2.806s] [ext: omni.kit.manipulator.prim.core-107.0.4] startup
[2.811s] [ext: omni.fabric.commands-1.1.5] startup
[2.813s] [ext: omni.physx.vehicle-106.1.9] startup
[2.826s] [ext: omni.kit.widget.extended_searchfield-1.0.28] startup
[2.833s] [ext: omni.kit.manipulator.prim.fabric-107.0.3] startup
[2.835s] [ext: omni.kit.manipulator.prim.usd-107.0.2] startup
[2.836s] [ext: omni.kit.widget.timeline-105.0.1] startup
[2.838s] [ext: omni.kit.window.commands-0.2.6] startup
[2.839s] [ext: omni.physx.camera-106.1.9] startup
[2.848s] [ext: omni.kit.manipulator.camera-105.0.5] startup
[2.850s] [ext: omni.kit.viewport.menubar.camera-107.0.2] startup
[2.853s] [ext: omni.kit.manipulator.prim-107.0.0] startup
[2.854s] [ext: omni.kit.manipulator.selection-106.0.1] startup
[2.855s] [ext: omni.kit.window.toolbar-1.6.1] startup
[2.856s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[2.858s] [ext: omni.kit.viewport.menubar.render-106.1.1] startup
[2.859s] [ext: omni.kit.viewport.menubar.settings-107.0.3] startup
[2.862s] [ext: omni.physx.cct-106.1.9] startup
[2.870s] [ext: omni.physx.graph-106.1.9] startup
[2.885s] [ext: omni.physx.supportui-106.1.9] startup
[2.899s] [ext: omni.physx.telemetry-106.1.9] startup
[2.902s] [ext: omni.graph.ui_nodes-1.26.0] startup
[2.907s] [ext: omni.kit.viewport.bundle-104.0.1] startup
[2.907s] [ext: omni.isaac.universal_robots-0.3.5] startup
[2.908s] [ext: omni.kit.window.console-0.2.13] startup
[2.922s] [ext: omni.kit.window.script_editor-1.7.6] startup
[2.924s] [ext: omni.kit.menu.create-1.0.16] startup
[2.926s] [ext: omni.graph.bundle.action-2.4.1] startup
[2.928s] [ext: omni.physx.bundle-106.1.9] startup
[2.929s] [ext: omni.rtx.window.settings-0.6.17] startup
[2.933s] [ext: omni.kit.window.status_bar-0.1.7] startup
[2.937s] [ext: omni.kit.window.title-1.1.5] startup
[2.940s] [ext: omni.replicator.isaac-1.15.2] startup
[2.958s] [ext: omni.replicator.replicator_yaml-2.0.6] startup
[2.974s] [ext: omni.rtx.settings.core-0.6.3] startup
[2.983s] [ext: omni.kit.viewport.menubar.lighting-106.0.2] startup
[2.990s] [ext: semantics.schema.editor-0.3.8] startup
[2.994s] [ext: semantics.schema.property-1.0.4] startup
[2.995s] [ext: omni.kit.viewport.rtx-104.0.1] startup
[2.995s] [ext: omni.isaac.utils-1.0.1] startup
[2.998s] [ext: omni.isaac.cloner-0.8.1] startup
[3.000s] [ext: omni.importer.urdf-1.14.1] startup
[3.029s] [ext: omni.kit.window.stats-0.1.6] startup
[3.033s] [ext: omni.isaac.sim.python-4.2.0] startup
[3.034s] Simulation App Starting
[4.413s] app ready
2024-10-18 16:42:13 [4,842ms] [Warning] [omni.kit.imgui_renderer.plugin] _createExtendCursor: No windowing.
2024-10-18 16:42:13 [4,842ms] [Warning] [omni.kit.imgui_renderer.plugin] _createExtendCursor: No windowing.
[5.088s] Simulation App Startup Complete
2024-10-18 16:42:16 [7,187ms] [Warning] [curobo] Running in headless mode: native
[7.263s] [ext: omni.kit.asset_converter-2.1.21] startup
[7.279s] [ext: omni.kit.tool.asset_importer-2.5.5] startup
[7.328s] [ext: omni.kit.widget.zoombar-1.0.5] startup
[7.330s] [ext: omni.kit.browser.core-2.3.11] startup
[7.333s] [ext: omni.kit.browser.folder.core-1.9.13] startup
[7.336s] [ext: omni.isaac.asset_browser-0.5.1] startup
[7.380s] [ext: omni.kit.livestream.core-4.3.6] startup
[7.383s] [ext: omni.kit.streamsdk.plugins-4.5.3] startup
[7.383s] [ext: omni.kit.livestream.native-4.1.0] startup
Warning: Possible version incompatibility. Attempting to load carb::cudainterop::CudaInterop with version v0.13 against v0.11.

Failed to open [/var/run/utmp]
Active user not found. Using default user [kiosk]Streaming server started.
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
main: thread_init: already added for thread
main: thread_init: already added for thread
main: thread_init: already added for thread
2024-10-18 16:42:26 [17,150ms] [Warning] [carb.livestream.plugin] onClientEventRaised: NVST_CCE_DISCONNECTED when m_connectionCount 0 != 1  <-- happens when I try to connect 
**** Click Play to start simulation *****

streaming client

any help is greatly appreciated, thank you!

update

I have solved this issue by making a few changes to the code which are mentioned here https://docs.omniverse.nvidia.com/isaacsim/latest/manual_standalone_python.html#simulationapp

these are the changes

from isaacsim import SimulationApp

# This sample enables a livestream server to connect to when running headless
CONFIG = {
    "width": 1280,
    "height": 720,
    "window_width": 1920,
    "window_height": 1080,
    "headless": True,
    "hide_ui": False,  # Show the GUI
    "renderer": "RayTracedLighting",
    "display_options": 3286,  # Set display options to show default grid
}

# Start the omniverse application
kit = SimulationApp(launch_config=CONFIG)

from omni.isaac.core.utils.extensions import enable_extension

# Default Livestream settings
kit.set_setting("/app/window/drawMouse", True)
kit.set_setting("/app/livestream/proto", "ws")
kit.set_setting("/ngx/enabled", False)

# Note: Only one livestream extension can be enabled at a time

# Enable Native Livestream extension
# Default App: Streaming Client from the Omniverse Launcher
enable_extension("omni.kit.streamsdk.plugins-3.2.1")
enable_extension("omni.kit.livestream.core-3.2.0")
enable_extension("omni.kit.livestream.native")

 #  
 # rest 
 #

 # replace original render script with this 
    while kit._app.is_running() and not kit.is_exiting():
        my_world.step(render=True)

 ## rest ##