[Proposal] Enabling Omniverse Streaming Client

Mayankm96 commented 1 year ago

Proposal

Omniverse Streaming Client is a lightweight application allowing Users to connect to Omniverse applications deployed on the network or over the Internet.

https://docs.omniverse.nvidia.com/app_streaming-client/app_streaming-client/overview.html

Motivation

It is quite useful for people who are developing on remote servers. Thus, it would help to have a pipeline to enable this functionality easily.

Pitch

To enable live stream, the user needs to load the default app experience in their standalone python scripts
After that, adding the following piece of code to the script should enable live streaming:

import carb
from omni.isaac.core.utils.extensions import enable_extension
from omni.isaac.core.utils.carb import set_carb_setting

# settings interface
carb_settings = carb.settings.get_settings()
# set values
set_carb_setting(carb_settings, "/app/livestream/enabled", True)
set_carb_setting(carb_settings, "/app/window/drawMouse", True)
set_carb_setting(carb_settings, "/app/livestream/proto", "ws")
set_carb_setting(carb_settings, "/app/livestream/websocket/framerate_limit", 120)
set_carb_setting(carb_settings, "/ngx/enabled", False)
# enable extensions for live stream
enable_extension("omni.kit.livestream.native")
enable_extension("omni.services.streaming.manager")

Additional context

OmniIsaacGymEnvs has a way to enable live stream easily form the command line.
OmniGibson has a nice way to handle remote operation by setting the REMOTE_ENABLED flag in the terminal (corresponding: PR).

Checklist

[x] I have checked that there is no similar issue in the repo (required)

neale commented 1 year ago

This worked out of the box for me in the demo environments :) In the Sb3 / rlgames training scripts I found that the livestreaming was extremely slow -> glacial depending on the task. Could be a networking issue, ill check again tomorrow.

I foresee that the video functionality in #63 will be most useful, since online rendering is mostly useful for demos or debugging.

Mayankm96 commented 1 year ago

Yes, training with live streaming enabled is the same as running without headless. This adds a bunch of overhead because multiple kit extensions are being stepped at every app update call.

From what I tested using the Isaac-Franka-Reach-v0 environment:

Running headless without viewport enabled: ~65,000 FPS
Running headless with viewport enabled: ~57,000 FPS
Running with GUI: ~13,000 FPS

hhansen-bdai commented 1 year ago

Hello,

I am currently attempting to get livestreaming working from some demo scripts after making the above addition, but am encountering segfaults whenever I actually attempt to connect the streaming client.

For context, I am running out of the IsaacSim Docker container on a Google Cloud VM. When I run runheadless.native.sh I am able to stream as expected. It seems the steps I'm running should be identical to yours, and I can't seem to find anything in the readout which specifies the problem, so I'm not sure why I get this mysterious segfault. I would appreciate your insight! Thank you.

Here is the relevant excerpt of play_arms.py (the rest is unchanged):

...
# launch omniverse app
config = {"headless": args_cli.headless}
simulation_app = SimulationApp(config)

import carb
from omni.isaac.core.utils.extensions import enable_extension
from omni.isaac.core.utils.carb import set_carb_setting

# settings interface
carb_settings = carb.settings.get_settings()
# set values
set_carb_setting(carb_settings, "/app/livestream/enabled", True)
set_carb_setting(carb_settings, "/app/window/drawMouse", True)
set_carb_setting(carb_settings, "/app/livestream/proto", "ws")
set_carb_setting(carb_settings, "/app/livestream/websocket/framerate_limit", 120)
set_carb_setting(carb_settings, "/ngx/enabled", False)
# enable extensions for live stream
enable_extension("omni.kit.livestream.native")
enable_extension("omni.services.streaming.manager")

"""Rest everything follows."""

import torch

import omni.isaac.core.utils.prims as prim_utils
from omni.isaac.core.simulation_context import SimulationContext
from omni.isaac.core.utils.viewports import set_camera_view

import omni.isaac.orbit.utils.kit as kit_utils
from omni.isaac.orbit.robots.config.franka import FRANKA_PANDA_ARM_WITH_PANDA_HAND_CFG
from omni.isaac.orbit.robots.config.universal_robots import UR10_CFG
from omni.isaac.orbit.robots.single_arm import SingleArmManipulator
from omni.isaac.orbit.utils.assets import ISAAC_NUCLEUS_DIR

"""
Main
"""

def main():
...

And here is the command, and the resulting terminal readout and error:

root@isaac-sim-1:/isaac-sim/orbit/source/standalone/demo# orbit -p play_arms.py --headless
[INFO] Using python from: /isaac-sim/python.sh                                                                                             
[Warning] [omni.isaac.kit.simulation_app] Modules: ['omni.isaac.kit.app_framework'] were loaded before SimulationApp was started and might not be loaded correctly.
[Warning] [omni.isaac.kit.simulation_app] Please check to make sure no extra omniverse or pxr modules are imported before the call to SimulationApp(...)
Starting kit application with the following args:  ['/isaac-sim/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/isaac-sim/apps/omni.isaac.sim.python.kit', '--/app/tokens/exe-path=/isaac-sim/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1280', '--/app/renderer/resolution/height=720', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/isaac-sim/exts', '--ext-folder', '/isaac-sim/apps', '--/physics/cudaDevice=0', '--portable', '--no-window', '--allow-root']
Passing the following args to the base kit application:  ['--headless']
[Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
[Info] [carb] Logging to file: /isaac-sim/kit/logs/Kit/Isaac-Sim/2022.2/kit_20230424_165813.log
2023-04-24 16:58:13 [18ms] [Warning] [omni.ext.plugin] [ext: omni.sensors.nv.lidar] Extensions config 'extension.toml' doesn't exist '/isaac-sim/exts/omni.sensors.nv.lidar' or '/isaac-sim/exts/omni.sensors.nv.lidar/config'
2023-04-24 16:58:13 [18ms] [Warning] [omni.ext.plugin] [ext: omni.sensors.nv.radar] Extensions config 'extension.toml' doesn't exist '/isaac-sim/exts/omni.sensors.nv.radar' or '/isaac-sim/exts/omni.sensors.nv.radar/config'
[0.181s] [ext: omni.stats-0.0.0] startup
[0.219s] [ext: omni.rtx.shadercache-1.0.0] startup
[0.232s] [ext: omni.assets.plugins-0.0.0] startup
[0.235s] [ext: omni.gpu_foundation-0.0.0] startup
[0.246s] [ext: carb.windowing.plugins-1.0.0] startup
2023-04-24 16:58:13 [234ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:13 [234ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
[0.247s] [ext: omni.kit.renderer.init-0.0.0] startup
2023-04-24 16:58:15 [2,457ms] [Warning] [carb.graphics-vulkan.plugin] No command queue family supports flags: 0x100, queue type: 3. No queues of this type will be created

|---------------------------------------------------------------------------------------------|
| Driver Version: 525.60.11     | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|---------------------------------------------------------------------------------------------|
| 0   | Tesla T4                         | Yes: 0 |     | 15606   MB | 10de      | 0          |
|     |                                  |        |     |            | 1eb8      | 50704a2b.. |
|=============================================================================================|
| OS: Linux isaac-sim-1, Version: 5.15.0-1032-gcp
| Processor:            Intel(R) Xeon(R) CPU @ 2.30GHz | Cores: Unknown | Logical: 8
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 30085 | Free Memory: 23908
| Total Page/Swap (MB): 0 | Free Page/Swap: 0
|---------------------------------------------------------------------------------------------|
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] ECC is enabled for device 0. This will reduce rendering performance.
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] 
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] -----------------------------------------------------------------------
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] !!!!! Local system validation failed! Incorrect configuration detected.
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin]       Summary below. Details above.
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] -----------------------------------------------------------------------
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] 
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin]  - ECC:                         FAILED
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin]  - IOMMU:                       OK
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] 
2023-04-24 16:58:16 [3,425ms] [Warning] [gpu.foundation.plugin] -----------------------------------------------------------------------
[4.913s] [ext: omni.kit.pipapi-0.0.0] startup
[4.930s] [ext: omni.kit.pip_archive-0.0.0] startup
[4.938s] [ext: omni.kit.loop-isaac-1.0.0] startup
[4.939s] [ext: omni.kit.async_engine-0.0.0] startup
[4.941s] [ext: omni.kit.test-0.0.0] startup
[5.020s] [ext: omni.usd.config-1.0.0] startup
[5.027s] [ext: omni.usd.libs-1.0.0] startup
[5.207s] [ext: omni.isaac.core_archive-2.0.1] startup
[5.403s] [ext: omni.pip.torch-1_13_1-0.1.4] startup
[5.407s] [ext: omni.isaac.ml_archive-1.1.0] startup
[5.469s] [ext: omni.client-0.1.1] startup
[5.481s] [ext: omni.appwindow-1.0.1] startup
2023-04-24 16:58:18 [5,469ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:18 [5,469ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
[5.484s] [ext: omni.kit.renderer.core-0.0.0] startup
2023-04-24 16:58:18 [5,473ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:18 [5,473ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
2023-04-24 16:58:18 [5,476ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:18 [5,476ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
[5.497s] [ext: omni.kit.renderer.capture-0.0.0] startup
[5.500s] [ext: omni.kit.renderer.imgui-0.0.0] startup
2023-04-24 16:58:18 [5,491ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:18 [5,491ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
2023-04-24 16:58:18 [5,492ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:18 [5,492ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
[5.717s] [ext: carb.audio-0.1.0] startup
[5.721s] [ext: omni.ui-2.14.4] startup
[5.735s] [ext: omni.uiaudio-1.0.0] startup
[5.737s] [ext: omni.kit.mainwindow-1.0.0] startup
[5.739s] [ext: omni.kit.uiapp-0.0.0] startup
[5.739s] [ext: omni.usd.schema.physics-1.0.0] startup
[5.797s] [ext: omni.usd.schema.isaac-1.1.0] startup
[5.818s] [ext: omni.usd.schema.semantics-0.0.0] startup
[5.832s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[5.843s] [ext: omni.usd.schema.forcefield-0.0.0] startup
[5.853s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[5.864s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[5.874s] [ext: omni.usd.schema.anim-0.0.0] startup
[5.960s] [ext: omni.usd.schema.audio-0.0.0] startup
[5.969s] [ext: omni.usd.schema.scene.visualization-1.0.0] startup
[5.971s] [ext: omni.usd.schema.physx-0.0.0] startup
[6.009s] [ext: omni.gpucompute.plugins-0.0.0] startup
[6.010s] [ext: omni.timeline-1.0.5] startup
[6.012s] [ext: omni.kit.search_core-1.0.2] startup
[6.014s] [ext: omni.hydra.scene_delegate-0.3.0] startup
[6.022s] [ext: omni.kit.widget.path_field-2.0.4] startup
[6.028s] [ext: omni.kit.widget.filebrowser-2.3.10] startup
[6.049s] [ext: omni.kit.widget.browser_bar-2.0.5] startup
[6.051s] [ext: omni.kit.window.popup_dialog-2.0.16] startup
[6.057s] [ext: omni.kit.actions.core-1.0.0] startup
[6.061s] [ext: omni.usd.core-1.0.4] startup
[6.064s] [ext: omni.kit.audiodeviceenum-1.0.0] startup
[6.066s] [ext: omni.kit.widget.nucleus_connector-1.0.3] startup
[6.068s] [ext: omni.kit.commands-1.4.5] startup
[6.075s] [ext: omni.hydra.usdrt_delegate-4.3.2] startup
[6.096s] [ext: omni.kit.notification_manager-1.0.5] startup
[6.102s] [ext: omni.usd-1.6.30] startup
[6.157s] [ext: omni.kit.widget.versioning-1.3.8] startup
[6.161s] [ext: omni.iray.libs-0.0.0] startup
[6.169s] [ext: omni.kit.window.filepicker-2.7.15] startup
[6.235s] [ext: omni.mdl.neuraylib-0.1.0] startup
[6.240s] [ext: omni.kit.collaboration.channel_manager-1.0.9] startup
[6.245s] [ext: omni.kit.window.file_importer-1.0.10] startup
[6.247s] [ext: omni.mdl-0.1.0] startup
[6.306s] [ext: omni.kit.usd.layers-2.0.11] startup
[6.318s] [ext: omni.kit.menu.utils-1.4.7] startup
[6.334s] [ext: omni.kit.material.library-1.3.21] startup
[6.343s] [ext: omni.kit.menu.create-1.0.8] startup
[6.346s] [ext: omni.kit.clipboard-1.0.0] startup
[6.347s] [ext: omni.kit.widget.graph-1.5.6-104_2] startup
[6.391s] [ext: omni.kit.context_menu-1.5.12] startup
[6.396s] [ext: omni.kit.window.extensions-1.1.1] startup
[6.404s] [ext: omni.inspect-1.0.1] startup
[6.410s] [ext: omni.kit.window.property-1.8.2] startup
[6.413s] [ext: omni.kit.primitive.mesh-1.0.8] startup
[6.420s] [ext: omni.graph.core-2.65.4] startup
[6.424s] [ext: omni.isaac.version-1.0.0] startup
[6.425s] [ext: omni.kit.stage_templates-1.1.13] startup
[6.428s] [ext: omni.kit.usd_undo-0.1.2] startup
[6.430s] [ext: omni.graph.tools-1.17.2] startup
[6.457s] [ext: omni.ui.scene-1.5.18] startup
[6.465s] [ext: omni.graph-1.50.2] startup
[6.544s] [ext: omni.ui_query-1.1.1] startup
[6.546s] [ext: omni.volume-0.1.0] startup
[6.549s] [ext: omni.kit.ui_test-1.2.9] startup
[6.552s] [ext: omni.hydra.rtx-0.1.0] startup
[6.562s] [ext: omni.kit.widget.searchfield-1.0.10] startup
[6.565s] [ext: omni.kit.graph.delegate.default-1.0.18] startup
[6.568s] [ext: omni.debugdraw-0.1.1] startup
[6.574s] [ext: omni.kit.graph.editor.core-1.4.7] startup
[6.579s] [ext: omni.hydra.pxr-1.1.2] startup
[6.587s] [ext: omni.activity.core-1.0.1] startup
[6.593s] [ext: omni.kit.hydra_texture-1.0.11] startup
[6.600s] [ext: omni.kit.viewport.legacy_gizmos-1.0.10] startup
[6.607s] [ext: omni.kit.viewport.registry-104.0.2] startup
[6.608s] [ext: omni.kit.widget.viewport-104.1.16] startup
[6.612s] [ext: omni.kit.window.drop_support-1.0.1] startup
[6.614s] [ext: omni.kit.window.file_exporter-1.0.10] startup
[6.615s] [ext: omni.hydra.engine.stats-1.0.0] startup
[6.621s] [ext: omni.kit.window.content_browser_registry-0.0.1] startup
[6.622s] [ext: omni.kit.window.file-1.3.32] startup
[6.627s] [ext: omni.kit.viewport.window-104.1.23] startup
[6.672s] [ext: omni.kit.widget.settings-1.0.1] startup
[6.675s] [ext: omni.kit.widget.stage-2.7.24] startup
[6.683s] [ext: omni.kit.window.content_browser-2.6.8] startup
[6.717s] [ext: omni.kit.viewport.utility-1.0.14] startup
[6.718s] [ext: omni.kit.window.preferences-1.3.8] startup
[6.827s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[6.830s] [ext: omni.kit.widget.prompt-1.0.5] startup
[6.832s] [ext: omni.kit.property.usd-3.18.17] startup
[6.848s] [ext: omni.kit.graph.delegate.modern-1.6.5] startup
[6.850s] [ext: omni.graph.ui-1.24.2] startup
[6.907s] [ext: omni.physics.tensors-0.1.0] startup
[6.919s] [ext: omni.kit.graph.widget.variables-2.0.4] startup
[6.921s] [ext: omni.kit.graph.usd.commands-1.2.0] startup
[6.923s] [ext: omni.convexdecomposition-104.2.4-5.1] startup
[6.927s] [ext: omni.graph.window.core-1.43.3] startup
[6.941s] [ext: omni.kit.widget.zoombar-1.0.4] startup
[6.943s] [ext: omni.kvdb-0.0.0] startup
[6.947s] [ext: omni.usdphysics-104.2.4-5.1] startup
[6.950s] [ext: omni.localcache-0.0.0] startup
[6.954s] [ext: omni.graph.scriptnode-0.10.0] startup
[6.958s] [ext: omni.physx-104.2.4-5.1] startup
[6.984s] [ext: omni.command.usd-1.0.2] startup
[6.989s] [ext: omni.physx.tensors-0.1.0] startup
[6.998s] [ext: omni.graph.window.generic-1.3.15] startup
[7.001s] [ext: omni.kit.browser.core-2.2.2] startup
[7.010s] [ext: omni.graph.action-1.31.1] startup
[7.022s] [ext: omni.kit.browser.folder.core-1.7.3] startup
[7.029s] [ext: omni.kit.window.cursor-1.1.1] startup
[7.030s] [ext: omni.kit.widget.spinner-1.0.4] startup
[7.033s] [ext: omni.kit.browser.sample-1.2.5] startup
[7.036s] [ext: omni.graph.nodes-1.48.3] startup
[7.055s] [ext: omni.kit.viewport.menubar.core-104.4.3] startup
[7.086s] [ext: omni.kit.numpy.common-0.1.0] startup
[7.090s] [ext: omni.warp-0.6.3] startup
Warp 0.6.3 initialized:
   CUDA Toolkit: 11.5, Driver: 12.0
   Devices:
     "cpu"    | x86_64
     "cuda:0" | Tesla T4 (sm_75)
   Kernel cache: /root/.cache/warp/0.6.3
[7.479s] [ext: omni.syntheticdata-0.2.4] startup
2023-04-24 16:58:20 [7,485ms] [Warning] [omni.kit.viewport.menubar.core.menu_item.viewport_menu_item] ViewportMenuItem order argument is deprecated, use order_setting_path
[7.507s] [ext: omni.isaac.dynamic_control-1.2.3] startup
[7.593s] [ext: omni.replicator.core-1.7.8] startup
2023-04-24 16:58:20 [7,723ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
[7.942s] [ext: omni.isaac.core-1.46.3] startup
[9.142s] [ext: omni.kit.manipulator.viewport-104.0.7] startup
[9.145s] [ext: omni.isaac.core_nodes-0.23.2] startup
[9.170s] [ext: omni.isaac.cloner-0.4.1] startup
[9.171s] [ext: omni.isaac.lula-1.2.1] startup
[9.186s] [ext: omni.graph.visualization.nodes-2.0.0] startup
[9.196s] [ext: omni.isaac.debug_draw-0.2.3] startup
[9.203s] [ext: omni.isaac.gym-0.3.3] startup
[9.204s] [ext: omni.graph.window.action-1.5.5] startup
[9.206s] [ext: omni.isaac.ui-0.8.0] startup
[9.212s] [ext: usdrt.scenegraph-4.3.2] startup
[9.240s] [ext: omni.isaac.motion_planning-0.2.2] startup
[9.253s] [ext: omni.isaac.surface_gripper-0.4.0] startup
[9.260s] [ext: omni.isaac.range_sensor-0.8.1] startup
[9.312s] [ext: omni.isaac.motion_generation-4.5.6] startup
[9.321s] [ext: omni.isaac.manipulators-1.1.0] startup
[9.324s] [ext: omni.isaac.sensor-5.6.3] startup
2023-04-24 16:58:22 [9,312ms] [Warning] [omni.ext.plugin] [ext: omni.isaac.sensor-5.6.3] Native plugin search dir contains several files: /isaac-sim/exts/omni.isaac.sensor/bin . It makes dir walk slow, replace with the exact path to the plugin instead of a wildcard.
[9.493s] [ext: omni.isaac.dofbot-0.3.0] startup
[9.495s] [ext: omni.isaac.occupancy_map-0.2.7] startup
[9.512s] [ext: omni.isaac.quadruped-1.3.0] startup
[9.962s] [ext: omni.isaac.scene_blox-0.1.0] startup
[9.964s] [ext: omni.isaac.synthetic_utils-0.4.3] startup
[9.967s] [ext: omni.isaac.universal_robots-0.3.2] startup
[9.969s] [ext: omni.isaac.isaac_sensor-2.0.0] startup
2023-04-24 16:58:23 [9,957ms] [Warning] [omni.isaac.isaac_sensor] omni.isaac.isaac_sensor is deprecated and was renamed; use omni.isaac.sensor
[9.971s] [ext: omni.kit.widget.live_session_management-1.1.3] startup
[9.975s] [ext: omni.kit.hotkeys.core-1.0.8] startup
[9.985s] [ext: omni.isaac.wheeled_robots-0.6.3] startup
[9.999s] [ext: omni.kit.widget.stage_icons-1.0.2] startup
[10.001s] [ext: omni.kit.widget.layers-1.6.11] startup
[10.019s] [ext: omni.kit.selection-0.1.0] startup
[10.022s] [ext: omni.isaac.mjcf-0.3.1] startup
[10.033s] [ext: omni.kit.window.stage-2.3.12] startup
[10.039s] [ext: omni.kit.property.layer-1.1.4] startup
[10.042s] [ext: omni.kit.menu.edit-1.1.6] startup
[10.046s] [ext: omni.kit.menu.file-1.1.4] startup
[10.049s] [ext: omni.kit.menu.stage-1.2.1] startup
[10.050s] [ext: omni.kit.property.audio-1.0.6] startup
[10.052s] [ext: omni.kit.property.camera-1.0.3] startup
[10.054s] [ext: omni.kit.property.geometry-1.2.2] startup
[10.057s] [ext: omni.kit.property.light-1.0.6] startup
[10.060s] [ext: omni.hydra.scene_api-0.1.1] startup
[10.068s] [ext: omni.kit.property.material-1.8.19] startup
[10.072s] [ext: omni.kit.property.render-1.1.0] startup
[10.074s] [ext: omni.kit.property.skel-1.0.1] startup
[10.076s] [ext: omni.kit.property.transform-1.3.5] startup
[10.082s] [ext: omni.kit.manipulator.transform-104.6.12] startup
[10.085s] [ext: omni.kit.widget.toolbar-1.4.0] startup
[10.097s] [ext: omni.kit.profiler.window-2.0.5] startup
[10.207s] [ext: omni.kit.property.bundle-1.2.6] startup
[10.209s] [ext: omni.kit.manipulator.selector-1.0.1] startup
[10.212s] [ext: omni.kit.manipulator.tool.snap-1.2.4] startup
[10.221s] [ext: omni.kit.window.toolbar-1.4.0] startup
[10.225s] [ext: omni.kit.property.isaac-0.2.0] startup
[10.227s] [ext: omni.kit.stage_column.payload-1.0.8] startup
[10.232s] [ext: omni.kit.manipulator.camera-104.0.23] startup
[10.233s] [ext: omni.kit.manipulator.prim-104.10.7] startup
[10.242s] [ext: omni.kit.manipulator.selection-104.0.6] startup
[10.242s] [ext: omni.kit.viewport.menubar.display-104.1.11] startup
[10.245s] [ext: omni.kit.viewport.menubar.camera-104.1.9] startup
[10.252s] [ext: omni.rtx.window.settings-0.6.2] startup
[10.262s] [ext: omni.kit.viewport.menubar.settings-104.3.5] startup
[10.269s] [ext: omni.kit.viewport.menubar.render-104.0.20] startup
[10.274s] [ext: omni.physx.commands-104.2.4-5.1] startup
[10.426s] [ext: omni.kit.stage_column.variant-1.0.9] startup
[10.429s] [ext: omni.hydra.pxr.settings-1.0.3] startup
[10.431s] [ext: omni.kit.viewport.bundle-104.0.2] startup
[10.431s] [ext: omni.physx.ui-104.2.4-5.1] startup
[10.515s] [ext: omni.kit.viewport.menubar.lighting-104.0.9] startup
[10.519s] [ext: omni.kit.viewport.pxr-104.0.1] startup
[10.519s] [ext: omni.kit.viewport.ready-1.0.2] startup
[10.521s] [ext: omni.physx.demos-104.2.4-5.1] startup
[10.526s] [ext: omni.kit.property.physx-0.1.0] startup
[10.607s] [ext: omni.isaac.urdf-0.5.9] startup
[10.654s] [ext: omni.kit.widget.highlight_label-1.0.0] startup
[10.656s] [ext: omni.physx.supportui-104.2.4-5.1] startup
[10.678s] [ext: omni.physx.vehicle-104.2.4-5.1] startup
[10.698s] [ext: omni.physx.cct-104.2.4-5.1] startup
[10.712s] [ext: omni.physx.camera-104.2.4-5.1] startup
[10.722s] [ext: omni.kit.actions.window-1.1.0] startup
[10.729s] [ext: omni.kit.window.commands-0.2.4] startup
[10.731s] [ext: omni.kit.window.console-0.2.3] startup
[10.744s] [ext: omni.physx.bundle-104.2.4-5.1] startup
[10.745s] [ext: omni.graph.bundle.action-1.3.0] startup
[10.745s] [ext: omni.kit.window.script_editor-1.7.2] startup
[10.748s] [ext: omni.kit.hotkeys.window-1.1.3] startup
[10.756s] [ext: omni.kit.window.status_bar-0.1.5] startup
[10.762s] [ext: omni.kit.window.title-1.1.2] startup
2023-04-24 16:58:23 [10,750ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:23 [10,750ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
2023-04-24 16:58:23 [10,752ms] [Warning] [carb] [Plugin: libomni.structuredlog.plugin.so] Module /isaac-sim/kit/libomni.structuredlog.plugin.so remained loaded after unload request
2023-04-24 16:58:23 [10,753ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed.
2023-04-24 16:58:23 [10,753ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
[10.768s] [ext: omni.replicator.composer-1.2.10] startup
[10.775s] [ext: omni.replicator.isaac-1.7.4] startup
[10.798s] [ext: omni.resourcemonitor-1.0.0] startup
[10.806s] [ext: omni.rtx.settings.core-0.5.8] startup
[10.814s] [ext: omni.isaac.franka-0.4.0] startup
[10.817s] [ext: omni.kit.viewport.rtx-104.0.0] startup
[10.817s] [ext: semantics.schema.editor-0.3.3] startup
[10.823s] [ext: omni.kit.widget.live-2.0.3] startup
2023-04-24 16:58:24 [10,813ms] [Warning] [omni.kit.widget.live.cache_state_menu] Unable to detect Omniverse Cache Server. Consider installing it for better IO performance.
[10.826s] [ext: omni.isaac.utils-0.2.4] startup
[10.832s] [ext: omni.isaac.kit-1.4.1] startup
[10.832s] [ext: omni.isaac.cortex-0.3.2] startup
[10.833s] [ext: omni.kit.window.stats-0.1.2] startup
[10.835s] [ext: omni.isaac.sim.python-2022.2.1] startup
[10.837s] Simulation App Starting
2023-04-24 16:58:24 [11,072ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : CUDA device 0 (Tesla T4): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
[15.065s] app ready
[15.340s] RTX ready
[15.501s] Simulation App Startup Complete
[15.525s] [ext: omni.kit.renderer.cuda_interop-1.0.0] startup
[15.558s] [ext: omni.kit.streamsdk.plugins-1.0.1] startup
[15.558s] [ext: omni.kit.livestream.core-0.0.0] startup
[15.562s] [ext: omni.kit.livestream.native-0.1.5] startup

Active user not found. Using default user [kiosk]Streaming server started.
[16.085s] [ext: omni.services.facilities.base-1.0.2] startup
[16.100s] [ext: omni.services.core-1.3.0] startup
[16.262s] [ext: omni.services.transport.server.base-1.1.0] startup
[16.264s] [ext: omni.services.transport.server.http-1.1.2] startup
[16.283s] [ext: omni.services.streaming.manager-0.3.6] startup
2023-04-24 16:58:29 [16,360ms] [Warning] [omni.isaac.core.utils.nucleus] /persistent/app/omniverse/mountedDrives setting not found
2023-04-24 16:58:29 [16,610ms] [Warning] [omni.isaac.core.utils.nucleus] /persistent/app/omniverse/mountedDrives setting not found
2023-04-24 16:58:29 [16,611ms] [Warning] [omni.isaac.core.utils.nucleus] /persistent/app/omniverse/mountedDrives setting not found
2023-04-24 16:58:29 [16,696ms] [Warning] [omni.isaac.core.utils.nucleus] /persistent/app/omniverse/mountedDrives setting not found
2023-04-24 16:58:30 [17,480ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /World/Table_1/Visuals.proto_TableGeom_id0 that has not been populated
[INFO]: Setup complete...
[INFO]: Resetting robots state...
[INFO]: [Step 000]: Flipping gripper command...
[INFO]: [Step 200]: Flipping gripper command...
[INFO]: [Step 400]: Flipping gripper command...
[INFO]: [Step 600]: Flipping gripper command...
[INFO]: [Step 800]: Flipping gripper command...
[INFO]: Resetting robots state...
[INFO]: [Step 000]: Flipping gripper command...
/isaac-sim/python.sh: line 41:  1089 Segmentation fault      (core dumped) $python_exe "$@" $args
There was an error running python

hhansen-bdai commented 1 year ago

I was able to get this to run using the script above after applying this fix to omni.isaac.sim.python.kit in the /apps folder: In line # 550, change to: “omni.isaac.quadruped” = {order = 10}

Source: https://forums.developer.nvidia.com/t/running-a-standalone-example-with-gui-in-docker-container/248147/6?u=hhansen

Mayankm96 commented 1 year ago

Thanks for finding this fix! It seems that it is better for Orbit to have its own set of "app" configuration files. Just to help us patch these kinds of issues easily.

hhansen-bdai commented 1 year ago

Thanks for finding this fix! It seems that it is better for Orbit to have its own set of "app" configuration files. Just to help us patch these kinds of issues easily.

Sure thing! And I totally agree, no reason to make patching more difficult than it needs to be.

hhansen-bdai commented 1 year ago

For anyone attempting this with ROS-- The client will crash on a connection attempt if you load the ros_bridge extension before omni.kit.livestream.native. If you load it afterwards, it will behave as expected.

Mayankm96 commented 11 months ago

Closing this issue as we have added livestream options on the devel branch. For more details, please check the documentation.

THSID2023 commented 8 months ago

Does this work with workstation version of nucleus? or is it just for enterprise version?

isaac-sim / IsaacLab