search

isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
1.9k stars 733 forks source link

[Bug Report] Unable to use RL Games on Isaac-Lift-Cube-Franka-v0 #197

Closed StoneT2000 closed 7 months ago

StoneT2000 commented 8 months ago

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug

Trying to run RL games training code one the lift cube task but I get a error here and am not sure what to do fix it.

Steps to reproduce

Expected 

./orbit.sh -p source/standalone/workflows/rl_games/train.py --task Isaac-Lift-Cube-Franka-v0 --headless

To run RL on the Isaac Lift Cube Franka-v0 task. However I get the following

[3.343s] [ext: omni.isaac.ui-0.13.5] startup
2024-01-10 21:24:00 [3,400ms] [Error] [omni.kit.app._impl] [py stderr]: /home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.pip.torch-2_0_1-2.0.2+105.1.lx64/torch-2-0-1/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, "__version__") or LooseVersion(

/home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.pip.torch-2_0_1-2.0.2+105.1.lx64/torch-2-0-1/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, "__version__") or LooseVersion(
[INFO]: Parsing configuration from: <class 'omni.isaac.orbit_tasks.manipulation.lift.config.franka.joint_pos_env_cfg.FrankaCubeLiftEnvCfg'>
2024-01-10 21:24:00 [4,001ms] [Error] [__main__] 'rl_games_cfg_entry_point'
2024-01-10 21:24:00 [4,001ms] [Error] [__main__] Traceback (most recent call last):
  File "/home/stao/work/external/Orbit/source/standalone/workflows/rl_games/train.py", line 141, in <module>
    main()
  File "/home/stao/work/external/Orbit/source/standalone/workflows/rl_games/train.py", line 71, in main
    agent_cfg = load_cfg_from_registry(args_cli.task, "rl_games_cfg_entry_point")
  File "/home/stao/work/external/Orbit/source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks/utils/parse_cfg.py", line 56, in load_cfg_from_registry
    cfg_entry_point = gym.spec(task_name).kwargs.pop(entry_point_key)
KeyError: 'rl_games_cfg_entry_point'

Full logs here:

[INFO] Using python from: /home/stao/miniforge3/envs/orbit/bin/python                                                                                                       
[INFO][AppLauncher]: Loading experience file: /home/stao/work/external/Orbit/_isaac_sim/apps/omni.isaac.sim.python.gym.headless.kit .
Starting kit application with the following args:  ['/home/stao/work/external/Orbit/_isaac_sim/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/home/stao/work/external/Orbit/_isaac_sim/apps/omni.isaac.sim.python.gym.headless.kit', '--/app/tokens/exe-path=/home/stao/work/external/Orbit/_isaac_sim/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True', '--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1280', '--/app/renderer/resolution/height=720', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/home/stao/work/external/Orbit/_isaac_sim/exts', '--ext-folder', '/home/stao/work/external/Orbit/_isaac_sim/apps', '--/physics/cudaDevice=0', '--portable', '--no-window']
Passing the following args to the base kit application:  ['--task', 'Isaac-Lift-Cube-Franka-v0', '--headless']
Loading user config located at: '/home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/data/Kit/Isaac-Sim/2023.1/user.config.json'
[Info] [carb] Logging to file: /home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/logs/Kit/Isaac-Sim/2023.1/kit_20240110_132356.log
2024-01-10 21:23:56 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
2024-01-10 21:23:56 [4ms] [Warning] [omni.ext.plugin] [ext: omni.kit.sequencer.core-103.4.0+105.0] Built using kit version: 105.0. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
2024-01-10 21:23:56 [4ms] [Warning] [omni.ext.plugin] [ext: omni.kit.sequencer.usd-103.4.0+105.0] Built using kit version: 105.0. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
2024-01-10 21:23:56 [5ms] [Warning] [omni.ext.plugin] [ext: omni.kit.widget.timeline-105.0.1+105.0] Built using kit version: 105.0. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
2024-01-10 21:23:56 [5ms] [Warning] [omni.ext.plugin] [ext: omni.kit.window.sequencer-103.4.2-dev.3+105.0] Built using kit version: 105.0. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
2024-01-10 21:23:56 [5ms] [Warning] [omni.ext.plugin] [ext: omni.paint.brush.attributes-1.3.1+105.0] Built using kit version: 105.0. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
2024-01-10 21:23:56 [6ms] [Warning] [omni.ext.plugin] [ext: omni.usd.schema.sequence-2.3.0+105.0.lx64.r.cp310] Built using kit version: 105.0. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
[0.027s] [ext: omni.kit.async_engine-0.0.0] startup
[0.154s] [ext: omni.assets.plugins-0.0.0] startup
[0.155s] [ext: omni.stats-0.0.0] startup
[0.156s] [ext: omni.client-1.0.1] startup
[0.160s] [ext: omni.gpu_foundation-0.0.0] startup
[0.165s] [ext: omni.rtx.shadercache.vulkan-1.0.0] startup
[0.166s] [ext: carb.windowing.plugins-1.0.0] startup
[0.171s] [ext: omni.kit.renderer.init-0.0.0] startup

|---------------------------------------------------------------------------------------------|
| Driver Version: 535.129.03    | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 4090          | Yes: 0 |     | 24564   MB | 10de      | 0          |
|     |                                  |        |     |            | 2684      | 2b0351f8.. |
|     |                                  |        |     |            | 1         |            |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 6.2.0-39-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12101004 (1.21.1.4)
| Processor: 12th Gen Intel(R) Core(TM) i9-12900KS | Cores: 16 | Logical: 32
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 64075 | Free Memory: 56581
| Total Page/Swap (MB): 0 | Free Page/Swap: 0
|---------------------------------------------------------------------------------------------|
[0.583s] [ext: omni.kit.pipapi-0.0.0] startup
[0.584s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.584s] [ext: omni.pip.compute-1.2.0] startup
[0.584s] [ext: omni.pip.torch-2_0_1-2.0.2] startup
[0.605s] [ext: omni.pip.cloud-1.0.1] startup
[0.606s] [ext: omni.isaac.core_archive-2.2.1] startup
[0.606s] [ext: omni.isaac.ml_archive-1.1.3] startup
[0.606s] [ext: omni.kit.telemetry-0.5.0] startup
[0.622s] [ext: omni.mtlx-0.1.0] startup
[0.622s] [ext: omni.usd.config-1.0.3] startup
[0.624s] [ext: omni.gpucompute.plugins-0.0.0] startup
[0.625s] [ext: omni.usd.libs-1.0.0] startup
[0.683s] [ext: omni.kit.loop-isaac-1.1.0] startup
[0.684s] [ext: omni.kit.test-0.0.0] startup
[0.684s] [ext: omni.usd.schema.physics-0.0.0] startup
[0.768s] [ext: omni.usd.schema.physx-0.0.0] startup
[0.787s] [ext: omni.usd.schema.anim-0.0.0] startup
[0.805s] [ext: omni.usd.schema.audio-0.0.0] startup
[0.808s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[0.809s] [ext: omni.usd.schema.semantics-0.0.0] startup
[0.811s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[0.814s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[0.817s] [ext: omni.kvdb-0.0.0] startup
[0.818s] [ext: omni.usd_resolver-1.0.1] startup
[0.821s] [ext: omni.localcache-0.0.0] startup
[0.822s] [ext: omni.usd.core-1.1.8] startup
[0.825s] [ext: omni.physx.foundation-105.1.12-5.1] startup
[0.825s] [ext: usdrt.scenegraph-7.2.34] startup
[0.849s] [ext: omni.hydra.scene_delegate-0.3.2] startup
[0.854s] [ext: omni.resourcemonitor-105.0.0] startup
[0.855s] [ext: omni.activity.core-1.0.1] startup
[0.856s] [ext: omni.appwindow-1.1.5] startup
[0.857s] [ext: omni.hydra.usdrt_delegate-7.2.34] startup
[0.866s] [ext: omni.kit.renderer.core-0.0.0] startup
[1.119s] [ext: omni.usdphysics-105.1.12-5.1] startup
[1.121s] [ext: omni.kit.renderer.capture-0.0.0] startup
[1.122s] [ext: omni.kit.numpy.common-0.1.2] startup
[1.184s] [ext: omni.kit.renderer.imgui-0.0.0] startup
[1.464s] [ext: omni.kit.actions.core-1.0.0] startup
[1.465s] [ext: omni.ui-2.18.5] startup
[1.471s] [ext: omni.graph.exec-0.3.0] startup
[1.472s] [ext: omni.kit.commands-1.4.6] startup
[1.474s] [ext: omni.kit.window.popup_dialog-2.0.23] startup
[1.476s] [ext: carb.audio-0.1.0] startup
[1.478s] [ext: omni.kit.exec.core-0.5.0] startup
[1.479s] [ext: omni.timeline-1.0.9] startup
[1.480s] [ext: omni.kit.widget.nucleus_connector-1.1.4] startup
[1.481s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[1.481s] [ext: omni.convexdecomposition-105.1.12-5.1] startup
[1.484s] [ext: omni.usd-1.10.18] startup
[1.519s] [ext: omni.physx.cooking-105.1.12-5.1] startup
[1.524s] [ext: omni.iray.libs-0.0.0] startup
[1.527s] [ext: omni.physx-105.1.12-5.1] startup
2024-01-10 21:23:58 [1,518ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
[1.534s] [ext: omni.mdl.neuraylib-0.2.0] startup
[1.535s] [ext: omni.isaac.dynamic_control-1.2.6] startup
[1.539s] [ext: omni.kit.widget.versioning-1.4.6] startup
[1.540s] [ext: omni.mdl-0.2.1] startup
[1.549s] [ext: omni.kit.helper.file_utils-0.1.6] startup
[1.559s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[1.559s] [ext: omni.kit.widget.path_field-2.0.8] startup
[1.560s] [ext: omni.kit.widget.filebrowser-2.3.35] startup
[1.562s] [ext: omni.kit.search_core-1.0.5] startup
[1.562s] [ext: omni.kit.widget.browser_bar-2.0.9] startup
[1.563s] [ext: omni.kit.widget.search_delegate-1.0.4] startup
[1.563s] [ext: omni.ui.scene-1.7.0] startup
[1.566s] [ext: omni.kit.notification_manager-1.0.6] startup
[1.567s] [ext: omni.kit.clipboard-1.0.3] startup
[1.567s] [ext: omni.isaac.kit-1.4.7] startup
[1.567s] [ext: omni.kit.window.filepicker-2.10.13] startup
[1.572s] [ext: omni.kit.usd.layers-2.1.27] startup
[1.577s] [ext: omni.physics.tensors-0.1.0] startup
[1.581s] [ext: omni.kit.window.file_importer-1.0.22] startup
[1.582s] [ext: omni.kit.menu.utils-1.5.6] startup
[1.587s] [ext: omni.physx.tensors-0.1.0] startup
[1.599s] [ext: omni.warp.core-1.0.0-beta.2] startup
Warp 1.0.0-beta.2 initialized:
   CUDA Toolkit: 11.5, Driver: 12.2
   Devices:
     "cpu"    | x86_64
     "cuda:0" | NVIDIA GeForce RTX 4090 (sm_89)
   Kernel cache: /home/stao/.cache/warp/1.0.0-beta.2
[1.658s] [ext: omni.kit.material.library-1.3.41] startup
[1.660s] [ext: omni.isaac.version-1.0.3] startup
[1.661s] [ext: omni.kit.window.title-1.1.3] startup
[1.661s] [ext: omni.isaac.core-3.3.2] startup
[2.247s] [ext: omni.isaac.cloner-0.7.2] startup
[2.247s] [ext: omni.physx.fabric-105.1.12-5.1] startup
[2.251s] [ext: omni.kit.mainwindow-1.0.1] startup
[2.251s] [ext: omni.isaac.gym-0.10.0] startup
[2.252s] [ext: omni.isaac.sim.python.gym.headless-2023.1.1] startup
[2.252s] Simulation App Starting
[2.370s] app ready
[2.721s] Simulation App Startup Complete
[2.727s] [ext: omni.kit.widget.graph-1.8.4] startup
[2.746s] [ext: omni.kit.widget.searchfield-1.1.3] startup
[2.746s] [ext: omni.kit.graph.delegate.default-1.1.1] startup
[2.747s] [ext: omni.inspect-1.0.1] startup
[2.748s] [ext: omni.kit.graph.editor.core-1.5.1] startup
[2.749s] [ext: omni.kit.graph.usd.commands-1.3.0] startup
[2.750s] [ext: omni.graph.core-2.139.12] startup
[2.753s] [ext: omni.kit.menu.create-1.0.11] startup
[2.757s] [ext: omni.kit.hotkeys.core-1.3.0] startup
[2.761s] [ext: omni.volume-0.4.0] startup
[2.763s] [ext: omni.kit.context_menu-1.6.8] startup
[2.764s] [ext: omni.kit.window.file_exporter-1.0.22] startup
[2.765s] [ext: omni.hydra.rtx-0.1.0] startup
[2.772s] [ext: omni.kit.widget.searchable_combobox-1.0.5] startup
[2.772s] [ext: omni.kit.widget.options_menu-1.0.5] startup
[2.773s] [ext: omni.kit.hydra_texture-1.1.12] startup
[2.776s] [ext: omni.kit.widget.settings-1.0.6] startup
[2.777s] [ext: omni.kit.widget.filter-1.1.2] startup
[2.777s] [ext: omni.kit.widget.material_preview-1.0.8] startup
[2.778s] [ext: omni.kit.window.preferences-1.3.23] startup
[2.783s] [ext: omni.kit.widget.stage-2.9.6] startup
[2.787s] [ext: omni.kit.usd_undo-0.1.2] startup
[2.787s] [ext: omni.graph.tools-1.41.3] startup
[2.796s] [ext: omni.kit.window.material_graph-1.8.11] startup
[2.815s] [ext: omni.graph-1.106.1] startup
[2.837s] [ext: omni.kit.primitive.mesh-1.0.15] startup
[2.842s] [ext: omni.kit.stage_templates-1.1.20] startup
[2.846s] [ext: omni.graph.action-1.55.1] startup
[2.932s] [ext: omni.graph.image.core-0.1.0] startup
[2.933s] [ext: omni.graph.nodes-1.106.3] startup
[3.043s] [ext: omni.kit.viewport.utility-1.0.16] startup
[3.045s] [ext: omni.graph.scriptnode-1.1.6] startup
[3.049s] [ext: omni.hydra.engine.stats-1.0.1] startup
[3.053s] [ext: omni.syntheticdata-0.4.10] startup
[3.067s] [ext: omni.warp-1.0.0-beta.2] startup
[3.109s] [ext: omni.command.usd-1.0.2] startup
[3.110s] [ext: omni.replicator.core-1.10.20] startup
2024-01-10 21:23:59 [3,107ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
[3.317s] [ext: omni.uiaudio-1.0.0] startup
[3.321s] [ext: omni.kit.uiapp-0.0.0] startup
[3.321s] [ext: omni.kit.widget.highlight_label-1.0.1] startup
[3.322s] [ext: omni.kit.window.property-1.9.6] startup
[3.327s] [ext: omni.ui_query-1.1.1] startup
[3.328s] [ext: omni.kit.ui_test-1.2.13] startup
[3.329s] [ext: omni.kit.window.extensions-1.3.0] startup
[3.336s] [ext: omni.kit.manipulator.viewport-104.0.9] startup
[3.337s] [ext: omni.kit.viewport.registry-104.0.5] startup
[3.337s] [ext: omni.graph.visualization.nodes-2.1.1] startup
2024-01-10 21:24:00 [3,322ms] [Warning] [omni.graph.visualization.nodes._impl.extension] No Viewport Window to add omni.graph.visualization.nodes-2.1.1 scene to
[3.343s] [ext: omni.isaac.ui-0.13.5] startup
2024-01-10 21:24:00 [3,400ms] [Error] [omni.kit.app._impl] [py stderr]: /home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.pip.torch-2_0_1-2.0.2+105.1.lx64/torch-2-0-1/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, "__version__") or LooseVersion(

/home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.pip.torch-2_0_1-2.0.2+105.1.lx64/torch-2-0-1/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, "__version__") or LooseVersion(
[INFO]: Parsing configuration from: <class 'omni.isaac.orbit_tasks.manipulation.lift.config.franka.joint_pos_env_cfg.FrankaCubeLiftEnvCfg'>
2024-01-10 21:24:00 [4,001ms] [Error] [__main__] 'rl_games_cfg_entry_point'
2024-01-10 21:24:00 [4,001ms] [Error] [__main__] Traceback (most recent call last):
  File "/home/stao/work/external/Orbit/source/standalone/workflows/rl_games/train.py", line 141, in <module>
    main()
  File "/home/stao/work/external/Orbit/source/standalone/workflows/rl_games/train.py", line 71, in main
    agent_cfg = load_cfg_from_registry(args_cli.task, "rl_games_cfg_entry_point")
  File "/home/stao/work/external/Orbit/source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks/utils/parse_cfg.py", line 56, in load_cfg_from_registry
    cfg_entry_point = gym.spec(task_name).kwargs.pop(entry_point_key)
KeyError: 'rl_games_cfg_entry_point'

[4.026s] Simulation App Shutting Down
2024-01-10 21:24:00 [4,023ms] [Warning] [omni.core.ITypeFactory] Module /home/stao/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.

System Info

Describe the characteristic of your environment:

ozhanozen commented 7 months ago

This error is due to the fact that the configuration entry point for the rl_games is indeed missing when the environment is being registered to the gym.

To add it, you can insert the keyword argument "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_ppo_cfg.yaml", somewhere between lines 22-25 here:

https://github.com/NVIDIA-Omniverse/orbit/blob/57e766cf68c942191a74e24269b780ee9a817535/source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks/manipulation/lift/config/franka/__init__.py#L18-L28

You need to do the same for all the other environment variations on the same file. This solved the issue for me, but I tested on the 939a768.

Mayankm96 commented 7 months ago

Thanks @ozhanozen for the answer!

Yes, in general, we provide the facility to train environments with different frameworks. However, it is intractable for us to tune all the frameworks for all the environments we provide. The PPO implementations vary, parameter names are not the same -- all of this makes it not so straightforward.

If you manage to tune them nicely, we are more than happy to welcome MRs with the right settings.