[Bug Report] Teleoperation not working in teleop example sim 2023.1.1

[mikelasa]

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug

The teleoperation example is not working (source/standalone/environments/teleoperation/teleop_se3_agent.py). Happens the same in this closed issue: https://github.com/NVIDIA-Omniverse/orbit/issues/46, the teleop only close and opens the gripper but the robot is not moving, just selecting different options from the left panel. I tried to use the solution of that issue, that is, using a big sensitivity with the keyboard and it doesn't work. Apart from that, I tried with the spacemouse compact too and the robot wont move, just close and open the gripper.

I created a new issue cause the solution of this issue and mine are slightly different (task differs, sim version...) when launching the demo:

• old issue: ./orbit.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Franka-v0 --num_envs 1 --cpu --device keyboard

• my launch: ./orbit.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --device keyboard --sensitivity 1000.0

Steps to reproduce

Just launch the teleop demo:

./orbit.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --device keyboard --sensitivity 1000.0

The terminal log

mikel@mikelpc:~/Desktop/IsaacSimControl/src/orbit$ ./orbit.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --device keyboard --sensitivity 1000.0 
[INFO] Using python from: /home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/python.sh                                                                                                                                                                                                                     
[INFO][AppLauncher]: Loading experience file: /home/mikel/Desktop/IsaacSimControl/src/orbit/source/apps/orbit.python.kit
Starting kit application with the following args:  ['/home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/home/mikel/Desktop/IsaacSimControl/src/orbit/source/apps/orbit.python.kit', '--/app/tokens/exe-path=/home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True', '--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1280', '--/app/renderer/resolution/height=720', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/exts', '--ext-folder', '/home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/apps', '--/physics/cudaDevice=0', '--portable']
Passing the following args to the base kit application:  ['--task', 'Isaac-Lift-Cube-Franka-IK-Rel-v0', '--num_envs', '1', '--device', 'keyboard', '--sensitivity', '1000.0']
Loading user config located at: '/home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/data/Kit/Isaac-Sim/2023.1/user.config.json'
[Info] [carb] Logging to file: /home/mikel/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/logs/Kit/Isaac-Sim/2023.1/kit_20240523_092942.log
2024-05-23 07:29:42 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
[0.050s] [ext: omni.kit.async_engine-0.0.0] startup
[0.332s] [ext: omni.activity.core-1.0.1] startup
[0.334s] [ext: omni.assets.plugins-0.0.0] startup
[0.335s] [ext: omni.stats-0.0.0] startup
[0.336s] [ext: omni.client-1.0.1] startup
[0.341s] [ext: omni.activity.profiler-1.0.2] startup
[0.343s] [ext: omni.gpu_foundation-0.0.0] startup
[0.352s] [ext: omni.rtx.shadercache.vulkan-1.0.0] startup
[0.353s] [ext: carb.windowing.plugins-1.0.0] startup
[0.360s] [ext: omni.kit.renderer.init-0.0.0] startup
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

2024-05-23 07:29:43 [565ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) UHD Graphics 630 (CFL GT2)
2024-05-23 07:29:43 [565ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) UHD Graphics 630 (CFL GT2)

|---------------------------------------------------------------------------------------------|
| Driver Version: 525.85.05     | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 2080          | Yes: 0 |     | 8438    MB | 10de      | 0          |
|     |                                  |        |     |            | 1e90      | 8567608b.. |
|     |                                  |        |     |            | 1         |            |
|---------------------------------------------------------------------------------------------|
| 1   | Intel(R) UHD Graphics 630 (CFL.. |        |     | 23919   MB | 8086      | 0          |
|     |                                  |        |     |            | 3e98      | ef3b2e4b.. |
|     |                                  |        |     |            | 0         |            |
|=============================================================================================|
| OS: 20.04.6 LTS (Focal Fossa) ubuntu, Version: 20.04.6, Kernel: 5.15.0-100-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12013000 (1.20.13.0)
| Processor: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | Cores: 8 | Logical: 16
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 31892 | Free Memory: 26823
| Total Page/Swap (MB): 2047 | Free Page/Swap: 2047
|---------------------------------------------------------------------------------------------|
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin] IOMMU is enabled. Found 26 items in /sys/kernel/iommu_groups/.
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin] An input-output memory management unit (IOMMU) appears to be enabled on this system.
On bare-metal Linux systems, CUDA and the display driver do not support IOMMU-enabled PCIe peer to peer memory copy.
If you are on a bare-metal Linux system, please disable the IOMMU. Otherwise you risk image corruption and program instability.
This typically can be controlled via BIOS settings (Intel Virtualization Technology for Directed I/O (VT-d) or AMD I/O Virtualization Technology (AMD-Vi)) and kernel parameters (iommu, intel_iommu, amd_iommu).
Note that in virtual machines with GPU pass-through (vGPU) the IOMMU needs to be enabled.
Since we can not reliably detect whether this system is bare-metal or a virtual machine, we show this warning in any case when an IOMMU appears to be enabled.
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin]  
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin] -----------------------------------------------------------------------
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin] !!!!! Local system validation failed! Incorrect configuration detected.
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin]       Summary below. Details above.
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin] -----------------------------------------------------------------------
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin]  
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin]  - ECC:                         OK
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin]  - IOMMU:                       FAILED
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin]  
2024-05-23 07:29:43 [802ms] [Warning] [gpu.foundation.plugin] -----------------------------------------------------------------------
[0.951s] [ext: omni.kit.pipapi-0.0.0] startup
[0.952s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.952s] [ext: omni.pip.compute-1.2.0] startup
[0.952s] [ext: omni.pip.torch-2_0_1-2.0.2] startup
[0.989s] [ext: omni.pip.cloud-1.0.1] startup
[0.992s] [ext: omni.isaac.core_archive-2.2.1] startup
[0.992s] [ext: omni.isaac.ml_archive-1.1.3] startup
[0.993s] [ext: omni.kit.loop-isaac-1.1.0] startup
[0.993s] [ext: omni.kit.test-0.0.0] startup
[0.994s] [ext: omni.kit.telemetry-0.5.0] startup
[1.006s] [ext: omni.mtlx-0.1.0] startup
[1.006s] [ext: omni.usd.config-1.0.3] startup
[1.010s] [ext: omni.gpucompute.plugins-0.0.0] startup
[1.010s] [ext: omni.usd.libs-1.0.0] startup
[1.105s] [ext: omni.appwindow-1.1.5] startup
[1.576s] [ext: omni.kit.renderer.core-0.0.0] startup
[2.423s] [ext: omni.kit.renderer.capture-0.0.0] startup
[2.425s] [ext: omni.kit.renderer.imgui-0.0.0] startup
[2.498s] [ext: carb.audio-0.1.0] startup
[2.519s] [ext: omni.ui-2.18.5] startup
[2.528s] [ext: omni.uiaudio-1.0.0] startup
[2.529s] [ext: omni.kit.mainwindow-1.0.1] startup
[2.530s] [ext: omni.kit.uiapp-0.0.0] startup
[2.530s] [ext: omni.usd.schema.physics-0.0.0] startup
[2.648s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[2.654s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[2.655s] [ext: omni.usd.schema.anim-0.0.0] startup
[2.675s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[2.677s] [ext: omni.usd.schema.forcefield-0.0.0] startup
[2.682s] [ext: omni.usd.schema.physx-0.0.0] startup
[2.707s] [ext: omni.usd.schema.semantics-0.0.0] startup
[2.709s] [ext: omni.usd.schema.audio-0.0.0] startup
[2.712s] [ext: omni.usd.schema.isaac-2.0.1] startup
[2.718s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[2.723s] [ext: omni.usd_resolver-1.0.1] startup
[2.727s] [ext: omni.resourcemonitor-105.0.0] startup
[2.729s] [ext: omni.usd.core-1.1.8] startup
[2.735s] [ext: omni.graph.exec-0.3.0] startup
[2.735s] [ext: omni.timeline-1.0.9] startup
[2.738s] [ext: omni.kit.actions.core-1.0.0] startup
[2.740s] [ext: omni.kit.window.popup_dialog-2.0.23] startup
[2.744s] [ext: omni.hydra.usdrt_delegate-7.2.34] startup
[2.758s] [ext: omni.hydra.scene_delegate-0.3.2] startup
[2.765s] [ext: omni.kit.commands-1.4.6] startup
[2.768s] [ext: omni.kit.widget.nucleus_connector-1.1.4] startup
[2.769s] [ext: omni.kit.exec.core-0.5.0] startup
[2.771s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[2.772s] [ext: omni.kit.collaboration.telemetry-1.0.0] startup
[2.772s] [ext: omni.usd-1.10.18] startup
[2.811s] [ext: omni.kit.collaboration.channel_manager-1.0.11] startup
[2.811s] [ext: omni.kit.usd.layers-2.1.27] startup
[2.819s] [ext: omni.kit.menu.utils-1.5.6] startup
[2.827s] [ext: omni.iray.libs-0.0.0] startup
[2.830s] [ext: omni.kit.primitive.mesh-1.0.15] startup
[2.833s] [ext: omni.mdl.neuraylib-0.2.0] startup
[2.834s] [ext: omni.kit.stage_templates-1.1.20] startup
[2.836s] [ext: omni.mdl-0.2.1] startup
[2.852s] [ext: omni.kit.viewport.registry-104.0.5] startup
[2.853s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[2.853s] [ext: omni.kit.helper.file_utils-0.1.6] startup
[2.871s] [ext: omni.kit.search_core-1.0.5] startup
[2.871s] [ext: omni.kit.widget.filebrowser-2.3.35] startup
[2.875s] [ext: omni.kit.widget.path_field-2.0.8] startup
[2.876s] [ext: omni.kit.notification_manager-1.0.6] startup
[2.877s] [ext: omni.kit.widget.browser_bar-2.0.9] startup
[2.878s] [ext: omni.ui.scene-1.7.0] startup
[2.882s] [ext: omni.inspect-1.0.1] startup
[2.883s] [ext: omni.kit.widget.search_delegate-1.0.4] startup
[2.884s] [ext: omni.kit.widget.versioning-1.4.6] startup
[2.886s] [ext: omni.hydra.engine.stats-1.0.1] startup
[2.890s] [ext: omni.graph.tools-1.41.3] startup
[2.904s] [ext: omni.graph.core-2.139.12] startup
[2.907s] [ext: omni.kit.clipboard-1.0.3] startup
[2.908s] [ext: omni.volume-0.4.0] startup
[2.910s] [ext: omni.kit.window.filepicker-2.10.13] startup
[2.918s] [ext: omni.hydra.rtx-0.1.0] startup
[2.928s] [ext: omni.hydra.pxr-1.1.6] startup
[2.932s] [ext: omni.kit.window.file_importer-1.0.22] startup
[2.934s] [ext: omni.kit.hydra_texture-1.1.12] startup
[2.938s] [ext: omni.kit.viewport.legacy_gizmos-1.0.14] startup
[2.942s] [ext: omni.kit.window.file_exporter-1.0.22] startup
[2.942s] [ext: omni.kit.window.drop_support-1.0.1] startup
[2.943s] [ext: omni.kit.widget.viewport-105.1.7] startup
[2.946s] [ext: omni.kit.material.library-1.3.41] startup
[2.950s] [ext: omni.kit.window.file-1.3.44] startup
[2.952s] [ext: omni.kit.viewport.window-105.1.12] startup
[2.969s] [ext: omni.command.usd-1.0.2] startup
[2.970s] [ext: omni.kit.viewport.utility-1.0.16] startup
[2.971s] [ext: omni.kit.selection-0.1.3] startup
[2.971s] [ext: usdrt.scenegraph-7.2.34] startup
[3.008s] [ext: omni.kit.usd_undo-0.1.2] startup
[3.008s] [ext: omni.kit.menu.edit-1.1.14] startup
[3.010s] [ext: omni.graph-1.106.1] startup
[3.109s] [ext: omni.kit.menu.file-1.1.7] startup
[3.111s] [ext: omni.isaac.kit-1.4.7] startup
[3.112s] [ext: omni.graph.action-1.55.1] startup
[3.159s] [ext: omni.kvdb-0.0.0] startup
[3.161s] [ext: omni.kit.collaboration.presence_layer-1.0.8] startup
[3.164s] [ext: omni.localcache-0.0.0] startup
[3.165s] [ext: omni.kit.widget.options_menu-1.0.5] startup
[3.167s] [ext: omni.physx.foundation-105.1.12-5.1] startup
[3.167s] [ext: omni.kit.menu.create-1.0.11] startup
[3.170s] [ext: omni.usdphysics-105.1.12-5.1] startup
[3.173s] [ext: omni.convexdecomposition-105.1.12-5.1] startup
[3.179s] [ext: omni.kit.widget.filter-1.1.2] startup
[3.180s] [ext: omni.kit.hotkeys.core-1.3.0] startup
[3.184s] [ext: omni.kit.context_menu-1.6.8] startup
[3.186s] [ext: omni.physx.cooking-105.1.12-5.1] startup
[3.194s] [ext: omni.kit.window.content_browser_registry-0.0.1] startup
[3.196s] [ext: omni.kit.widget.stage-2.9.6] startup
[3.205s] [ext: omni.kit.widget.stage_icons-1.0.4] startup
[3.206s] [ext: omni.physics.tensors-0.1.0] startup
[3.214s] [ext: omni.physx-105.1.12-5.1] startup
2024-05-23 07:29:46 [3,214ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
[3.227s] [ext: omni.kit.window.content_browser-2.7.17] startup
[3.239s] [ext: omni.kit.numpy.common-0.1.2] startup
[3.241s] [ext: omni.kit.window.stage-2.4.4] startup
[3.246s] [ext: omni.warp.core-1.0.0-beta.2] startup
Warp 1.0.0-beta.2 initialized:
   CUDA Toolkit: 11.5, Driver: 12.0
   Devices:
     "cpu"    | x86_64
     "cuda:0" | NVIDIA GeForce RTX 2080 (sm_75)
   Kernel cache: /home/mikel/.cache/warp/1.0.0-beta.2
[3.377s] [ext: omni.physx.tensors-0.1.0] startup
[3.383s] [ext: omni.isaac.version-1.0.3] startup
[3.384s] [ext: omni.isaac.dynamic_control-1.2.6] startup
[3.394s] [ext: omni.kit.menu.stage-1.2.3] startup
[3.397s] [ext: omni.isaac.core-3.3.2] startup
[4.435s] [ext: omni.kit.widget.highlight_label-1.0.1] startup
[4.436s] [ext: omni.kit.widget.searchable_combobox-1.0.5] startup
[4.437s] [ext: omni.kit.widget.searchfield-1.1.3] startup
[4.439s] [ext: omni.kit.widget.graph-1.8.4] startup
[4.444s] [ext: omni.ui_query-1.1.1] startup
[4.446s] [ext: omni.kit.manipulator.viewport-104.0.9] startup
[4.448s] [ext: omni.kit.widget.settings-1.0.6] startup
[4.450s] [ext: omni.kit.window.property-1.9.6] startup
[4.454s] [ext: omni.kit.window.extensions-1.3.0] startup
[4.460s] [ext: omni.kit.ui_test-1.2.13] startup
[4.462s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[4.475s] [ext: omni.kit.profiler.window-2.1.2] startup
2024-05-23 07:29:47 [4,602ms] [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
[4.615s] [ext: omni.isaac.ui-0.13.5] startup
[4.624s] [ext: omni.kit.window.preferences-1.3.23] startup
[4.630s] [ext: omni.kit.widget.prompt-1.0.6] startup
[4.631s] [ext: omni.isaac.utils-0.6.0] startup
[4.643s] [ext: omni.kit.widget.live_session_management-1.2.22] startup
[4.648s] [ext: omni.isaac.cloner-0.7.2] startup
[4.650s] [ext: omni.kit.widget.layers-1.6.32] startup
[4.661s] [ext: omni.kit.raycast.query-1.0.1] startup
[4.671s] [ext: omni.kit.widget.toolbar-1.5.6] startup
[4.679s] [ext: omni.kit.manipulator.transform-104.6.15] startup
[4.681s] [ext: omni.kit.widget.spinner-1.0.6] startup
[4.684s] [ext: omni.kit.window.cursor-1.1.2] startup
[4.685s] [ext: omni.kit.manipulator.tool.snap-1.4.2] startup
[4.692s] [ext: omni.kit.manipulator.selector-1.0.1] startup
[4.694s] [ext: omni.kit.viewport.menubar.core-105.0.19] startup
[4.707s] [ext: omni.kit.viewport.actions-105.1.6] startup
[4.714s] [ext: omni.kit.manipulator.prim-105.0.11] startup
[4.720s] [ext: omni.kit.viewport.menubar.camera-105.1.9] startup
[4.727s] [ext: omni.kit.viewport.menubar.display-105.1.0] startup
[4.729s] [ext: omni.kit.viewport.menubar.render-105.1.1] startup
[4.733s] [ext: omni.kit.viewport.menubar.settings-105.0.9] startup
[4.737s] [ext: omni.kit.manipulator.selection-104.0.8] startup
[4.739s] [ext: omni.kit.manipulator.camera-105.0.4] startup
[4.742s] [ext: omni.rtx.window.settings-0.6.9] startup
[4.748s] [ext: semantics.schema.editor-0.3.4] startup
[4.751s] [ext: omni.kit.viewport.bundle-104.0.1] startup
[4.752s] [ext: omni.hydra.pxr.settings-1.0.5] startup
[4.754s] [ext: omni.activity.pump-1.0.0] startup
[4.755s] [ext: omni.kit.property.layer-1.1.5] startup
[4.758s] [ext: omni.kit.property.usd-3.21.15] startup
[4.768s] [ext: omni.kit.viewport.menubar.lighting-105.0.7] startup
[4.772s] [ext: omni.kit.viewport.pxr-104.0.1] startup
[4.772s] [ext: omni.kit.viewport.ready-1.0.5] startup
[4.774s] [ext: omni.kit.viewport.rtx-104.0.0] startup
[4.774s] [ext: omni.kit.property.audio-1.0.9] startup
[4.779s] [ext: omni.kit.widget.live-2.1.6] startup
[4.781s] [ext: omni.debugdraw-0.1.2] startup
[4.787s] [ext: omni.kit.property.camera-1.0.5] startup
[4.789s] [ext: omni.kit.property.light-1.0.8] startup
[4.792s] [ext: omni.kit.property.geometry-1.2.11] startup
[4.796s] [ext: omni.kit.widget.cache_indicator-2.0.8] startup
[4.798s] [ext: omni.usdphysics.ui-105.1.12-5.1] startup
[4.831s] [ext: omni.physx.commands-105.1.12-5.1] startup
[4.835s] [ext: omni.kit.property.skel-1.0.4] startup
[4.838s] [ext: omni.kit.property.render-1.1.1] startup
[4.843s] [ext: omni.kit.window.commands-0.2.4] startup
[4.845s] [ext: omni.physx.ui-105.1.12-5.1] startup
[4.876s] [ext: omni.hydra.scene_api-0.1.2] startup
[4.884s] [ext: usdrt.gf-1.0.2] startup
[4.884s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[4.886s] [ext: omni.kit.property.material-1.8.36] startup
[4.890s] [ext: omni.kit.property.transform-1.3.10] startup
[4.895s] [ext: omni.physx.demos-105.1.12-5.1] startup
[4.899s] [ext: omni.kit.property.physx-0.1.0] startup
[4.964s] [ext: omni.kit.property.bundle-1.2.9] startup
[4.966s] [ext: omni.kit.actions.window-1.1.0] startup
[4.970s] [ext: omni.kit.window.console-0.2.10] startup
[4.980s] [ext: omni.kit.window.script_editor-1.7.4] startup
[4.983s] [ext: omni.physx.camera-105.1.12-5.1] startup
[4.992s] [ext: omni.kit.property.isaac-0.2.2] startup
[4.995s] [ext: omni.kit.hotkeys.window-1.4.1] startup
[5.000s] [ext: omni.isaac.debug_draw-0.6.1] startup
2024-05-23 07:29:47 [4,995ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2024-05-23 07:29:47 [4,995ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
[5.010s] [ext: omni.physx.graph-105.1.12-5.1] startup
[5.023s] [ext: omni.kit.window.toolbar-1.5.4] startup
[5.026s] [ext: omni.kit.widget.zoombar-1.0.5] startup
[5.027s] [ext: omni.physx.cct-105.1.12-5.1] startup
[5.043s] [ext: omni.physx.vehicle-105.1.12-5.1] startup
[5.057s] [ext: omni.physx.supportui-105.1.12-5.1] startup
[5.073s] [ext: omni.kit.browser.core-2.3.10] startup
[5.078s] [ext: omni.kit.window.status_bar-0.1.6] startup
[5.084s] [ext: omni.kit.window.title-1.1.3] startup
[5.085s] [ext: omni.physx.bundle-105.1.12-5.1] startup
[5.085s] [ext: omni.kit.browser.folder.core-1.9.7] startup
[5.090s] [ext: omni.kit.graph.delegate.default-1.1.1] startup
[5.091s] [ext: omni.graph.image.core-0.1.0] startup
[5.092s] [ext: omni.graph.nodes-1.106.3] startup
[5.266s] [ext: omni.kit.browser.sample-1.4.6] startup
[5.269s] [ext: omni.kit.widget.material_preview-1.0.8] startup
[5.272s] [ext: omni.kit.graph.usd.commands-1.3.0] startup
[5.274s] [ext: omni.kit.graph.editor.core-1.5.1] startup
[5.276s] [ext: omni.kit.renderer.cuda_interop-1.0.1] startup
[5.279s] [ext: omni.syntheticdata-0.4.10] startup
[5.311s] [ext: omni.warp-1.0.0-beta.2] startup
[5.385s] [ext: omni.kit.window.material_graph-1.8.11] startup
[5.402s] [ext: omni.graph.scriptnode-1.1.6] startup
[5.410s] [ext: omni.kit.stage_column.payload-1.0.12] startup
[5.413s] [ext: omni.kit.livestream.core-2.3.0] startup
[5.415s] [ext: omni.kit.streamsdk.plugins-2.5.2] startup
[5.415s] [ext: omni.replicator.core-1.10.20] startup
2024-05-23 07:29:48 [5,427ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
[5.610s] [ext: semantics.schema.property-1.0.2] startup
[5.613s] [ext: omni.rtx.settings.core-0.5.13] startup
[5.619s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[5.621s] [ext: omni.kit.menu.common-1.1.4] startup
[5.624s] [ext: omni.kit.livestream.native-2.4.0] startup

Active user not found. Using default user [kiosk]Streaming server started.
[5.717s] [ext: omni.kit.window.stats-0.1.4] startup
[5.719s] [ext: orbit.python-2023.1.1] startup
[5.720s] Simulation App Starting
2024-05-23 07:29:48 [5,732ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/ecoMode/enabled has been requested but is not available
2024-05-23 07:29:48 [5,732ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/directLighting/enabled has been requested but is not available
2024-05-23 07:29:48 [5,732ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/reflections/enabled has been requested but is not available
2024-05-23 07:29:48 [5,733ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/translucency/enabled has been requested but is not available
2024-05-23 07:29:48 [5,733ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/raytracing/subsurface/enabled has been requested but is not available
2024-05-23 07:29:48 [5,733ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/caustics/enabled has been requested but is not available
2024-05-23 07:29:49 [7,063ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2024-05-23 07:29:49 [7,063ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[8.036s] app ready
2024-05-23 07:29:51 [8,336ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2024-05-23 07:29:51 [8,336ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[8.532s] RTX ready
[8.878s] Simulation App Startup Complete
[8.963s] [ext: omni.kit.stagerecorder.core-105.0.4] startup
[INFO]: Parsing configuration from: <class 'omni.isaac.orbit_tasks.manipulation.lift.config.franka.ik_rel_env_cfg.FrankaCubeLiftEnvCfg'>
[10.755s] [ext: omni.physx.fabric-105.1.12-5.1] startup
[INFO]: Base environment:
    Environment device    : cuda:0
    Physics step-size     : 0.01
    Rendering step-size   : 0.01
    Environment step-size : 0.02
    Physics GPU pipeline  : True
    Physics GPU simulation: True
[INFO]: Time taken for scene creation : 2.882599 seconds
[INFO]: Scene manager:  <class InteractiveScene>
    Number of environments: 1
    Environment spacing   : 2.5
    Source prim name      : /World/envs/env_0
    Global prim paths     : []
    Replicate physics     : True
[INFO]: Starting the simulation. This may take a few seconds. Please wait...
2024-05-23 07:29:57 [15,056ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /World/envs/env_0/Object/visuals.proto_visuals_id0 that has not been populated
~*~*~*
~*~*~* Direct GPU Helper:
~*~*~*   1 rigid dynamic
~*~*~*   1 articulation, maxLinks=11
~*~*~*
[INFO]: Time taken for simulation start : 1.615682 seconds
[INFO] Command Manager:  <CommandManager> contains 1 active terms.
+------------------------------------------+
|           Active Command Terms           |
+-------+-------------+--------------------+
| Index | Name        |        Type        |
+-------+-------------+--------------------+
|   0   | object_pose | UniformPoseCommand |
+-------+-------------+--------------------+

[INFO] Action Manager:  <ActionManager> contains 2 active terms.
+--------------------------------------+
|    Active Action Terms (shape: 7)    |
+-------+------------------+-----------+
| Index | Name             | Dimension |
+-------+------------------+-----------+
|   0   | body_joint_pos   |         6 |
|   1   | finger_joint_pos |         1 |
+-------+------------------+-----------+

[INFO] Observation Manager: <ObservationManager> contains 1 groups.
+-----------------------------------------------------------+
| Active Observation Terms in Group: 'policy' (shape: (35,)) |
+----------+-------------------------------------+----------+
|  Index   | Name                                |  Shape   |
+----------+-------------------------------------+----------+
|    0     | joint_pos                           |   (9,)   |
|    1     | joint_vel                           |   (9,)   |
|    2     | object_position                     |   (3,)   |
|    3     | target_object_position              |   (7,)   |
|    4     | actions                             |   (7,)   |
+----------+-------------------------------------+----------+

[INFO] Event Manager:  <EventManager> contains 1 active terms.
+--------------------------------------+
| Active Event Terms in Mode: 'reset'  |
+---------+----------------------------+
|  Index  | Name                       |
+---------+----------------------------+
|    0    | reset_all                  |
|    1    | reset_object_position      |
+---------+----------------------------+

[INFO] Termination Manager:  <TerminationManager> contains 1 active terms.
+------------------------------------+
|      Active Termination Terms      |
+-------+-----------------+----------+
| Index | Name            | Time Out |
+-------+-----------------+----------+
|   0   | object_dropping |  False   |
+-------+-----------------+----------+

[INFO] Reward Manager:  <RewardManager> contains 6 active terms.
+-----------------------------------------------------+
|                 Active Reward Terms                 |
+-------+-----------------------------------+---------+
| Index | Name                              |  Weight |
+-------+-----------------------------------+---------+
|   0   | reaching_object                   |     1.0 |
|   1   | lifting_object                    |    15.0 |
|   2   | object_goal_tracking              |    16.0 |
|   3   | object_goal_tracking_fine_grained |     5.0 |
|   4   | action_rate                       |  -0.001 |
|   5   | joint_vel                         | -0.0001 |
+-------+-----------------------------------+---------+

[INFO] Curriculum Manager:  <CurriculumManager> contains 2 active terms.
+--------------------------+
| Active Curriculum Terms  |
+---------+----------------+
|  Index  | Name           |
+---------+----------------+
|    0    | action_rate    |
|    1    | joint_vel      |
+---------+----------------+

Creating window for environment.
[INFO]: Completed setting up the environment...
Keyboard Controller for SE(3): Se3Keyboard
    Keyboard name: Isaac Sim Python 105.1.2+release.133510.b82c1e1e.tc
    ----------------------------------------------
    Toggle gripper (open/close): K
    Move arm along x-axis: W/S
    Move arm along y-axis: A/D
    Move arm along z-axis: Q/E
    Rotate arm along x-axis: Z/X
    Rotate arm along y-axis: T/G
    Rotate arm along z-axis: C/V
[29.997s] Simulation is stopped. The app will keep running with physics disabled. Press Ctrl+C or close the window to exit the app.
[31.189s] Simulation App Shutting Down

System Info

Describe the characteristic of your environment:

• Commit: f71340ebbe10cdbf7aa50735329ebebdc97e3e97
• Isaac Sim Version: 2023.1.1
• OS: Ubuntu 20.04
• GPU: RTX 2080 Super
• CUDA: 12.0
• GPU Driver: 525.85.05

Additional context

I'm running the simulations on a laptop too (alienware area 51m r1). The previous reported bugs were: https://github.com/NVIDIA-Omniverse/Orbit/issues/1 https://github.com/NVIDIA-Omniverse/orbit/issues/46

Checklist

• [x] I have checked that there is no similar issue in the repo (required)
• [x] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

• [x] moving the robot

[mikelasa]

I found the issue. I implemented a torque controller and thus, disabled the PD gains from the franka model that the teleop tutorial needs for the IK controller that uses.

The solution was to restore the original values