search

isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
2.34k stars 969 forks source link

Imported urdf loads only the stage tree but not the robot #530

Closed mikelasa closed 5 months ago

mikelasa commented 5 months ago

Hello,

I'm unable to load my own panda model, which is a copy of the FRANKA_PANDA_HIGH_PD_CFG robot. Before updating to issac lab the articulation was loaded and worked but now I cant. The strange thing is that the simulation launches without problems but only loads the stage tree (robot2), as it can be seen here: robot roots

Apart from this problem, after I updated the repository this morning to the latest commit (9ab6b48) the bin is also loaded in the stage tree but no in the simulation.

This is the code for the interactive scene:


from omni.isaac.lab_assets.franka import FRANKA_PANDA_HIGH_PD_CFG  # isort: skip
import omni.isaac.lab.sim as sim_utils
from omni.isaac.lab.assets import AssetBaseCfg, RigidObjectCfg

from omni.isaac.lab.envs import ManagerBasedEnvCfg
from omni.isaac.lab.managers import ObservationGroupCfg as ObsGroup
from omni.isaac.lab.managers import ObservationTermCfg as ObsTerm
from omni.isaac.lab.managers import EventTermCfg as EventTerm
from omni.isaac.lab.scene import InteractiveSceneCfg
from omni.isaac.lab.utils.assets import ISAAC_NUCLEUS_DIR
from omni.isaac.lab.utils import configclass
from omni.isaac.lab.managers import SceneEntityCfg
from omni.isaac.lab.actuators import ImplicitActuatorCfg
from omni.isaac.lab.assets.articulation import ArticulationCfg

import omni.isaac.lab.envs.mdp as mdp```
@configclass
class TableTopSceneCfg(InteractiveSceneCfg):
    """Configuration for a cart-pole scene."""

    # ground plane
    ground = AssetBaseCfg(
        prim_path="/World/defaultGroundPlane",
        spawn=sim_utils.GroundPlaneCfg(),
        init_state=AssetBaseCfg.InitialStateCfg(pos=(0.0, 0.0, -1.05)),
    )

    # lights
    dome_light = AssetBaseCfg(
        prim_path="/World/Light", spawn=sim_utils.DomeLightCfg(intensity=3000.0, color=(0.75, 0.75, 0.75))
    )

    # mount
    table = AssetBaseCfg(
        prim_path="{ENV_REGEX_NS}/Table",
        spawn=sim_utils.UsdFileCfg(
            usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Mounts/SeattleLabTable/table_instanceable.usd", scale=(2, 2, 1.0)
        ),
        init_state=AssetBaseCfg.InitialStateCfg(
            pos=(0.5, 0.5, 0.0),
            rot=(0.7071, 0.0, 0.0, 0.7071)
        ),
    )

    # bin
    bin: RigidObjectCfg = RigidObjectCfg(
        prim_path="{ENV_REGEX_NS}/Bin",
        spawn=sim_utils.UsdFileCfg(
            usd_path="http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Props/KLT_Bin/small_KLT.usd",
            scale=(1.5, 1.5, 1.5)
        ),
        init_state=RigidObjectCfg.InitialStateCfg(
            pos=(0.35, 0.4, 0.1),
            rot=(0.0, 0.0, 0.0, 0.0)
        ),
    )

    # robot 1
    robot1: ArticulationCfg = FRANKA_PANDA_HIGH_PD_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot1")
    robot1.actuators["panda_shoulder"].stiffness = 0.0
    robot1.actuators["panda_shoulder"].damping = 0.0
    robot1.actuators["panda_forearm"].stiffness = 0.0
    robot1.actuators["panda_forearm"].damping = 0.0
    robot1.init_state.pos = (0.0, 0.0, 0.0)

    robot2: ArticulationCfg = ArticulationCfg(
        prim_path="{ENV_REGEX_NS}/Robot2",
        spawn=sim_utils.UsdFileCfg(
            usd_path="/home/mikel/IsaacOrbitLFD/IsaacLab/source/extensions/omni.isaac.lab_assets/data/Robots/franka/panda.usd",
            activate_contact_sensors=False,
            rigid_props=sim_utils.RigidBodyPropertiesCfg(
                disable_gravity=True,
                max_depenetration_velocity=5.0,
            ),
            articulation_props=sim_utils.ArticulationRootPropertiesCfg(
                enabled_self_collisions=True,
                solver_position_iteration_count=8,
                solver_velocity_iteration_count=0,
                fix_root_link=True
            ),
            #collision_props=sim_utils.CollisionPropertiesCfg(contact_offset=0.005, rest_offset=0.0),
        ),
        init_state=ArticulationCfg.InitialStateCfg(
            joint_pos={
                "panda_joint1": 0.0,
                "panda_joint2": -0.569,
                "panda_joint3": 0.0,
                "panda_joint4": -2.810,
                "panda_joint5": 0.0,
                "panda_joint6": 3.037,
                "panda_joint7": 0.741,
                "panda_finger_joint.*": 0.04,
            },
        ),
        actuators={
            "panda_shoulder": ImplicitActuatorCfg(
                joint_names_expr=["panda_joint[1-4]"],
                effort_limit=87.0,
                velocity_limit=2.175,
                stiffness=80.0,
                damping=4.0,
            ),
            "panda_forearm": ImplicitActuatorCfg(
                joint_names_expr=["panda_joint[5-7]"],
                effort_limit=12.0,
                velocity_limit=2.61,
                stiffness=80.0,
                damping=4.0,
            ),
            "panda_hand": ImplicitActuatorCfg(
                joint_names_expr=["panda_finger_joint.*"],
                effort_limit=200.0,
                velocity_limit=0.2,
                stiffness=2e3,
                damping=1e2,
            ),
        },
        soft_joint_pos_limit_factor=1.0,
    )
    robot2.actuators["panda_shoulder"].stiffness = 0
    robot2.actuators["panda_forearm"].stiffness = 0
    robot2.actuators["panda_shoulder"].damping = 0
    robot2.actuators["panda_forearm"].damping = 0
    robot2.init_state.pos = (0.0, 1.0, 0.0)
    robot2.init_state.rot = (0.0, 0.0, 0.0, 0.0)

The log: 

[INFO] Using python from: /home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/python.sh                                                                          
[INFO][AppLauncher]: Loading experience file: /home/mikel/IsaacOrbitLFD/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/app/../../../../../../apps/isaaclab.python.kit
Starting kit application with the following args:  ['/home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/home/mikel/IsaacOrbitLFD/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/app/../../../../../../apps/isaaclab.python.kit', '--/app/tokens/exe-path=/home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True', '--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1280', '--/app/renderer/resolution/height=720', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/exts', '--ext-folder', '/home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/apps', '--/renderer/activeGpu=0', '--/physics/cudaDevice=0', '--portable', '--/app/window/hideUi=False']
Passing the following args to the base kit application:  ['--num_envs', '1', '--command_mode', '2', '--impedance_mode', '1', '--sensitivity', '15', '--device', 'spacemouse']
Loading user config located at: '/home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/kit/data/Kit/Isaac-Sim/4.0/user.config.json'
[Info] [carb] Logging to file: /home/mikel/.local/share/ov/pkg/isaac-sim-4.0.0/kit/logs/Kit/Isaac-Sim/4.0/kit_20240618_142243.log
2024-06-18 12:22:43 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
[0.054s] [ext: omni.kit.async_engine-0.0.0] startup
[0.362s] [ext: omni.stats-0.0.0] startup
[0.362s] [ext: omni.client-1.1.0] startup
[0.388s] [ext: omni.datastore-0.0.0] startup
[0.388s] [ext: omni.blobkey-1.1.0] startup
[0.389s] [ext: omni.ujitso.default-1.0.0] startup
[0.389s] [ext: omni.hsscclient-0.0.0] startup
[0.390s] [ext: omni.rtx.shadercache.vulkan-1.0.0] startup
[0.391s] [ext: omni.assets.plugins-0.0.0] startup
[0.392s] [ext: omni.gpu_foundation-0.0.0] startup
[0.400s] [ext: carb.windowing.plugins-1.0.0] startup
[0.407s] [ext: omni.kit.renderer.init-0.0.0] startup
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

2024-06-18 12:22:43 [580ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) UHD Graphics 630 (CFL GT2)
2024-06-18 12:22:43 [581ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) UHD Graphics 630 (CFL GT2)

|---------------------------------------------------------------------------------------------|
| Driver Version: 525.85.05     | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 2080          | Yes: 0 |     | 8438    MB | 10de      | 0          |
|     |                                  |        |     |            | 1e90      | 8567608b.. |
|     |                                  |        |     |            | 1         |            |
|---------------------------------------------------------------------------------------------|
| 1   | Intel(R) UHD Graphics 630 (CFL.. |        |     | 23919   MB | 8086      | 0          |
|     |                                  |        |     |            | 3e98      | ef3b2e4b.. |
|     |                                  |        |     |            | 0         |            |
|=============================================================================================|
| OS: 20.04.6 LTS (Focal Fossa) ubuntu, Version: 20.04.6, Kernel: 5.15.0-100-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12013000 (1.20.13.0)
| Processor: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | Cores: 8 | Logical: 16
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 31892 | Free Memory: 25957
| Total Page/Swap (MB): 2047 | Free Page/Swap: 2047
|---------------------------------------------------------------------------------------------|
2024-06-18 12:22:44 [840ms] [Warning] [gpu.foundation.plugin] IOMMU is enabled.
[0.996s] [ext: omni.kit.pipapi-0.0.0] startup
[0.997s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.997s] [ext: omni.pip.compute-1.3.1] startup
[0.997s] [ext: omni.pip.cloud-1.1.3] startup
[1.000s] [ext: omni.isaac.core_archive-2.3.0] startup
[1.000s] [ext: omni.mtlx-0.1.0] startup
[1.000s] [ext: omni.usd.config-1.0.3] startup
[1.004s] [ext: omni.gpucompute.plugins-0.0.0] startup
[1.004s] [ext: omni.usd.libs-1.0.0] startup
[1.101s] [ext: omni.isaac.ml_archive-2.0.1] startup
[1.101s] [ext: omni.kit.loop-isaac-1.2.0] startup
[1.101s] [ext: omni.kit.test-0.0.0] startup
[1.141s] [ext: omni.kit.telemetry-0.5.0] startup
[1.158s] [ext: omni.appwindow-1.1.8] startup
[1.508s] [ext: omni.kit.renderer.core-1.0.1] startup
[2.264s] [ext: omni.kit.renderer.capture-0.0.0] startup
[2.265s] [ext: omni.kit.renderer.imgui-1.0.1] startup
[2.332s] [ext: omni.ui-2.23.11] startup
[2.342s] [ext: omni.kit.mainwindow-1.0.3] startup
[2.343s] [ext: carb.audio-0.1.0] startup
[2.363s] [ext: omni.uiaudio-1.0.0] startup
[2.364s] [ext: omni.kit.uiapp-0.0.0] startup
[2.364s] [ext: omni.usd.schema.semantics-0.0.0] startup
[2.476s] [ext: omni.usd.schema.isaac-2.0.2] startup
[2.482s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[2.483s] [ext: omni.usd.schema.audio-0.0.0] startup
[2.486s] [ext: omni.usd.schema.forcefield-106.0.16] startup
[2.506s] [ext: omni.usd.schema.physx-106.0.16] startup
[2.532s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[2.538s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[2.540s] [ext: omni.usd.schema.anim-0.0.0] startup
[2.555s] [ext: omni.kit.window.popup_dialog-2.0.24] startup
[2.559s] [ext: omni.usd_resolver-1.0.0] startup
[2.563s] [ext: omni.kit.widget.nucleus_connector-1.1.5] startup
[2.564s] [ext: omni.timeline-1.0.10] startup
[2.566s] [ext: omni.usd.core-1.2.11] startup
[2.569s] [ext: omni.kit.usd_undo-0.1.8] startup
[2.570s] [ext: omni.hydra.scene_delegate-0.3.3] startup
[2.576s] [ext: omni.activity.core-1.0.1] startup
[2.578s] [ext: omni.resourcemonitor-105.0.1] startup
[2.579s] [ext: omni.graph.exec-0.9.3] startup
[2.580s] [ext: omni.kit.actions.core-1.0.0] startup
[2.581s] [ext: usdrt.scenegraph-7.4.8] startup
[2.621s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[2.623s] [ext: omni.kit.exec.core-0.13.0] startup
[2.624s] [ext: omni.kit.commands-1.4.9] startup
[2.627s] [ext: omni.hydra.usdrt_delegate-7.4.7] startup
[2.645s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[2.646s] [ext: omni.usd-1.10.39] startup
[2.690s] [ext: omni.kit.search_core-1.0.5] startup
[2.691s] [ext: omni.kit.helper.file_utils-0.1.7] startup
[2.692s] [ext: omni.kit.widget.options_menu-1.1.4] startup
[2.695s] [ext: omni.kit.widget.filebrowser-2.3.47] startup
[2.699s] [ext: omni.kit.widget.options_button-1.0.2] startup
[2.700s] [ext: omni.kit.widget.search_delegate-1.0.4] startup
[2.701s] [ext: omni.kit.widget.context_menu-1.2.1] startup
[2.702s] [ext: omni.kit.usd.layers-2.1.30] startup
[2.710s] [ext: omni.kit.hotkeys.core-1.3.3] startup
[2.713s] [ext: omni.kit.widget.path_field-2.0.9] startup
[2.714s] [ext: omni.kit.clipboard-1.0.3] startup
[2.715s] [ext: omni.kit.menu.utils-1.5.23] startup
[2.723s] [ext: omni.ui.scene-1.9.3] startup
[2.728s] [ext: omni.kit.widget.browser_bar-2.0.9] startup
[2.729s] [ext: omni.kit.notification_manager-1.0.8] startup
[2.731s] [ext: omni.kit.stage_template.core-1.1.20] startup
[2.731s] [ext: omni.kit.primitive.mesh-1.0.16] startup
[2.734s] [ext: omni.kit.window.filepicker-2.10.32] startup
[2.744s] [ext: omni.kit.stage_templates-1.2.2] startup
[2.746s] [ext: omni.kit.window.file_importer-1.1.11] startup
[2.748s] [ext: omni.kit.window.file_exporter-1.0.29] startup
[2.749s] [ext: omni.kit.window.content_browser_registry-0.0.4] startup
[2.749s] [ext: omni.kit.widget.searchable_combobox-1.0.5] startup
[2.750s] [ext: omni.kit.window.file-1.3.52] startup
[2.753s] [ext: omni.kit.window.drop_support-1.0.1] startup
[2.753s] [ext: omni.kit.widget.filter-1.1.3] startup
[2.754s] [ext: omni.kit.widget.settings-1.1.0] startup
[2.755s] [ext: omni.kit.window.content_browser-2.9.13] startup
[2.769s] [ext: omni.kit.window.preferences-1.5.1] startup
[2.775s] [ext: omni.hydra.engine.stats-1.0.1] startup
[2.779s] [ext: omni.iray.libs-0.0.0] startup
[2.783s] [ext: omni.ujitso.processor.texture-1.0.0] startup
[2.783s] [ext: omni.mdl.neuraylib-0.2.4] startup
[2.785s] [ext: omni.inspect-1.0.1] startup
[2.786s] [ext: omni.volume-0.5.0] startup
[2.788s] [ext: omni.ujitso.client-0.0.0] startup
[2.788s] [ext: omni.graph.tools-1.77.0] startup
[2.807s] [ext: omni.graph.core-2.170.0] startup
[2.811s] [ext: omni.hydra.rtx-0.2.0] startup
[2.824s] [ext: omni.graph-1.135.0] startup
[2.934s] [ext: omni.kit.raycast.query-1.0.5] startup
[2.942s] [ext: omni.kit.hydra_texture-1.2.6] startup
[2.947s] [ext: omni.mdl-52.0.1] startup
[2.965s] [ext: omni.kit.context_menu-1.8.0] startup
[2.967s] [ext: omni.kit.widget.viewport-106.0.2] startup
[2.971s] [ext: omni.kit.viewport.registry-104.0.6] startup
[2.972s] [ext: omni.kit.material.library-1.4.0] startup
[2.978s] [ext: omni.convexdecomposition-106.0.16] startup
[2.982s] [ext: omni.kit.viewport.window-106.0.8] startup
[2.998s] [ext: omni.kit.widget.graph-1.12.6] startup
[3.003s] [ext: omni.kit.viewport.utility-1.0.17] startup
[3.005s] [ext: omni.kit.widget.prompt-1.0.7] startup
[3.006s] [ext: omni.kit.collaboration.presence_layer-1.0.8] startup
[3.008s] [ext: omni.kit.widget.highlight_label-1.0.1] startup
[3.010s] [ext: omni.kit.widget.searchfield-1.1.5] startup
[3.011s] [ext: omni.kit.widget.live_session_management.ui-1.0.1] startup
[3.015s] [ext: omni.physx.foundation-106.0.16] startup
[3.016s] [ext: omni.kvdb-106.0.16] startup
[3.017s] [ext: omni.kit.window.property-1.11.1] startup
[3.019s] [ext: omni.kit.widget.stage-2.10.26] startup
[3.029s] [ext: omni.kit.widget.live_session_management-1.2.18] startup
[3.032s] [ext: omni.usdphysics-106.0.16] startup
[3.035s] [ext: omni.localcache-106.0.16] startup
[3.036s] [ext: omni.kit.property.usd-3.21.26] startup
[3.045s] [ext: omni.physx.cooking-106.0.16] startup
[3.057s] [ext: omni.kit.window.extensions-1.4.7] startup
[3.062s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[3.064s] [ext: omni.graph.action_core-1.1.4] startup
[3.071s] [ext: omni.graph.ui-1.67.1] startup
[3.089s] [ext: omni.physx-106.0.16] startup
[3.101s] [ext: omni.kit.manipulator.camera-105.0.5] startup
[3.105s] [ext: omni.graph.action_nodes-1.21.3] startup
[3.114s] [ext: omni.physx.stageupdate-106.0.16] startup
[3.118s] [ext: omni.kit.numpy.common-0.1.2] startup
[3.120s] [ext: omni.graph.action-1.101.1] startup
[3.121s] [ext: omni.isaac.kit-1.10.0] startup
[3.123s] [ext: omni.isaac.dynamic_control-1.3.7] startup
[3.133s] [ext: omni.kit.graph.delegate.default-1.2.2] startup
[3.137s] [ext: omni.physics.tensors-106.0.16] startup
[3.145s] [ext: omni.isaac.version-1.1.0] startup
[3.146s] [ext: omni.warp.core-1.1.0] startup
Warp 1.1.0 initialized:
   CUDA Toolkit 11.5, Driver 12.0
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA GeForce RTX 2080" (8 GiB, sm_75, mempool enabled)
   Kernel cache:
     /home/mikel/.cache/warp/1.1.0
[3.254s] [ext: omni.physx.tensors-106.0.16] startup
[3.262s] [ext: omni.isaac.nucleus-0.1.2] startup
[3.263s] [ext: omni.kit.graph.delegate.modern-1.10.6] startup
[3.265s] [ext: omni.isaac.core-3.12.0] startup
[4.315s] [ext: omni.kit.graph.editor.core-1.5.3] startup
[4.319s] [ext: omni.kit.manipulator.viewport-107.0.0] startup
[4.321s] [ext: omni.kit.graph.widget.variables-2.1.0] startup
[4.323s] [ext: omni.kit.graph.usd.commands-1.3.1] startup
[4.325s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[4.332s] [ext: omni.graph.window.core-1.107.1] startup
[4.349s] [ext: omni.ui_query-1.1.2] startup
[4.353s] [ext: omni.graph.window.generic-1.23.1] startup
[4.361s] [ext: omni.kit.ui_test-1.2.18] startup
[4.365s] [ext: omni.kit.ui.actions-1.0.1] startup
[4.366s] [ext: omni.isaac.ui-0.15.1] startup
[4.370s] [ext: omni.kit.window.cursor-1.1.2] startup
[4.373s] [ext: omni.kit.menu.common-1.1.5] startup
[4.377s] [ext: omni.command.usd-1.0.3] startup
[4.378s] [ext: omni.isaac.menu-0.4.1] startup
[4.384s] [ext: omni.kit.viewport.menubar.core-107.0.0] startup
[4.404s] [ext: omni.kit.viewport.actions-105.2.8] startup
[4.409s] [ext: omni.graph.window.action-1.25.2] startup
[4.417s] [ext: omni.kit.viewport.menubar.display-106.0.0] startup
[4.420s] [ext: omni.debugdraw-0.1.3] startup
[4.425s] [ext: omni.kit.widget.toolbar-1.6.2] startup
[4.432s] [ext: omni.kit.manipulator.transform-104.7.5] startup
[4.436s] [ext: omni.usdphysics.ui-106.0.16] startup
[4.462s] [ext: omni.physx.commands-106.0.16] startup
[4.466s] [ext: omni.kit.manipulator.tool.snap-1.4.5] startup
[4.471s] [ext: omni.kit.manipulator.selector-1.1.1] startup
[4.474s] [ext: omni.physx.ui-106.0.16] startup
[4.502s] [ext: omni.kit.property.material-1.9.3] startup
[4.506s] [ext: omni.kit.viewport.manipulator.transform-106.0.0] startup
ViewportTransformManipulatorExt on_startup
[4.508s] [ext: omni.physx.demos-106.0.16] startup
[4.528s] [ext: omni.kit.property.physx-106.0.16] startup
[4.593s] [ext: omni.kit.manipulator.prim.core-106.0.2] startup
[4.601s] [ext: omni.fabric.commands-1.1.4] startup
[4.607s] [ext: omni.graph.image.core-0.3.2] startup
[4.611s] [ext: omni.physx.vehicle-106.0.16] startup
[4.628s] [ext: omni.kit.manipulator.prim.usd-106.0.1] startup
[4.630s] [ext: omni.kit.manipulator.prim.fabric-106.0.1] startup
[4.633s] [ext: omni.graph.image.nodes-1.0.2] startup
[4.634s] [ext: omni.physx.camera-106.0.16] startup
[4.644s] [ext: omni.kit.manipulator.prim-106.0.0] startup
[4.646s] [ext: omni.graph.nodes-1.141.2] startup
[4.656s] [ext: omni.graph.ui_nodes-1.24.1] startup
[4.662s] [ext: omni.kit.manipulator.selection-104.0.9] startup
[4.666s] [ext: omni.kit.widget.layers-1.7.9] startup
[4.702s] [ext: omni.kit.window.toolbar-1.6.1] startup
[4.718s] [ext: omni.graph.scriptnode-1.18.2] startup
[4.724s] [ext: omni.graph.bundle.action-2.0.4] startup
[4.731s] [ext: omni.physx.cct-106.0.16] startup
[4.761s] [ext: omni.physx.graph-106.0.16] startup
[4.787s] [ext: omni.physx.supportui-106.0.16] startup
[4.823s] [ext: omni.physx.telemetry-106.0.16] startup
[4.835s] [ext: omni.kit.widget.material_preview-1.0.16] startup
[4.845s] [ext: omni.sensors.tiled-0.0.3] startup
[4.851s] [ext: omni.syntheticdata-0.6.7] startup
[4.936s] [ext: omni.warp-1.1.0] startup
[4.993s] [ext: omni.physx.bundle-106.0.16] startup
[5.005s] [ext: omni.kit.window.material_graph-1.8.15] startup
[5.046s] [ext: omni.isaac.debug_draw-0.7.1] startup
[5.051s] [ext: omni.sensors.nv.common-1.0.1] startup
[5.071s] [ext: omni.replicator.core-1.11.8] startup
2024-06-18 12:22:48 [5,087ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
[5.225s] [ext: omni.kit.menu.create-1.0.12] startup
[5.227s] [ext: omni.sensors.nv.materials-1.0.0] startup
[5.231s] [ext: omni.isaac.range_sensor-3.0.1] startup
[5.265s] [ext: omni.isaac.core_nodes-1.15.4] startup
[5.287s] [ext: omni.sensors.nv.wpm-1.0.0] startup
[5.290s] [ext: omni.kit.actions.window-1.1.1] startup
[5.314s] [ext: omni.kit.selection-0.1.4] startup
[5.319s] [ext: omni.sensors.nv.radar-1.0.1] startup
[5.324s] [ext: omni.sensors.nv.lidar-1.0.1] startup
[5.333s] [ext: omni.kit.menu.file-1.1.9] startup
[5.338s] [ext: omni.kit.menu.edit-1.1.19] startup
[5.346s] [ext: omni.isaac.sensor-11.3.0] startup
[5.467s] [ext: omni.kit.hotkeys.window-1.4.5] startup
[5.481s] [ext: omni.kit.property.isaac-0.2.3] startup
[5.486s] [ext: omni.kit.property.layer-1.1.5] startup
[5.494s] [ext: omni.kit.stage_column.payload-2.0.0] startup
[5.499s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[5.504s] [ext: omni.kit.stagerecorder.core-105.0.5] startup
[5.518s] [ext: omni.rtx.window.settings-0.6.15] startup
[5.529s] [ext: omni.kit.widget.live-2.1.6] startup
[5.532s] [ext: omni.kit.viewport.menubar.camera-105.1.8] startup
[5.538s] [ext: omni.kit.viewport.menubar.render-106.1.2] startup
[5.541s] [ext: omni.kit.viewport.menubar.settings-106.0.0] startup
[5.545s] [ext: omni.kit.viewport.rtx-104.0.1] startup
[5.545s] [ext: omni.kit.widget.cache_indicator-2.0.8] startup
[5.594s] [ext: omni.kit.property.audio-1.0.11] startup
[5.597s] [ext: omni.kit.widget.calendar-1.0.8] startup
[5.599s] [ext: omni.kit.widget.stage_icons-1.0.3] startup
[5.601s] [ext: omni.hydra.scene_api-0.1.2] startup
[5.608s] [ext: omni.kit.property.geometry-1.3.0] startup
[5.613s] [ext: omni.kit.property.camera-1.0.6] startup
[5.615s] [ext: omni.kit.widget.extended_searchfield-1.0.27] startup
[5.623s] [ext: omni.kit.window.stage-2.5.9] startup
[5.629s] [ext: omni.kit.property.transform-1.5.1] startup
[5.633s] [ext: omni.kit.property.render-1.1.1] startup
[5.634s] [ext: omni.kit.property.light-1.0.8] startup
[5.638s] [ext: omni.kit.window.commands-0.2.5] startup
[5.640s] [ext: omni.kit.widget.timeline-105.0.1] startup
[5.642s] [ext: omni.kit.menu.stage-1.2.5] startup
[5.644s] [ext: omni.kit.property.bundle-1.2.11] startup
[5.647s] [ext: omni.kit.window.status_bar-0.1.6] startup
[5.651s] [ext: omni.kit.window.title-1.1.3] startup
[5.653s] [ext: omni.replicator.isaac-1.15.0] startup
[5.669s] [ext: omni.replicator.replicator_yaml-2.0.5] startup
[5.687s] [ext: omni.rtx.settings.core-0.6.0] startup
[5.695s] [ext: omni.kit.window.console-0.2.12] startup
[5.707s] [ext: semantics.schema.editor-0.3.4] startup
[5.711s] [ext: semantics.schema.property-1.0.2] startup
[5.716s] [ext: omni.kit.window.script_editor-1.7.6] startup
[5.721s] [ext: omni.kit.profiler.window-2.2.1] startup
2024-06-18 12:22:49 [5,711ms] [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
[5.732s] [ext: omni.isaac.utils-1.0.1] startup
[5.737s] [ext: omni.isaac.cloner-0.8.1] startup
[5.740s] [ext: omni.kit.window.stats-0.1.6] startup
[5.742s] [ext: isaaclab.python-1.0.0] startup
[5.747s] Simulation App Starting
2024-06-18 12:22:49 [5,868ms] [Warning] [rtx.neuraylib.plugin] [CUDA:RENDER]   0.1   CUDA   rend warn : CUDA module initialization failed.
2024-06-18 12:22:49 [5,868ms] [Warning] [rtx.neuraylib.plugin] [CUDA:RENDER]   0.1   CUDA   rend warn : The version of your CUDA driver is 12.0, but 12.2 is the required minimum
2024-06-18 12:22:49 [5,868ms] [Warning] [rtx.neuraylib.plugin] [CUDA:RENDER]   0.1   CUDA   rend warn : Please update your display driver (current version 525.85.5) (www.nvidia.com) to at least 535.104.05.
2024-06-18 12:22:49 [5,954ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : Your NVIDIA driver supports CUDA version up to 12.0; iray photoreal requires CUDA version 12.2; iray photoreal can only run in CPU mode. Please update your NVIDIA driver (www.nvidia.com) to at least 535.104.05.
2024-06-18 12:22:49 [5,954ms] [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER]   1.1   IRAY   rend warn : There is no CUDA-capable GPU available to the iray photoreal renderer.
[7.659s] app ready
[8.767s] Simulation App Startup Complete
[10.382s] [ext: omni.physx.fabric-106.0.16] startup
[INFO]: Base environment:
    Environment device    : cuda:0
    Physics step-size     : 0.001
    Rendering step-size   : 0.001
    Environment step-size : 0.004
    Physics GPU pipeline  : True
    Physics GPU simulation: True
[INFO]: Time taken for scene creation : 5.441967 seconds
[INFO]: Scene manager:  <class InteractiveScene>
    Number of environments: 1
    Environment spacing   : 2.5
    Source prim name      : /World/envs/env_0
    Global prim paths     : []
    Replicate physics     : True
[INFO]: Starting the simulation. This may take a few seconds. Please wait...
2024-06-18 12:23:00 [17,505ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /World/envs/env_0/Robot2/panda_rightfinger/visuals.proto_mesh_0_id0 that has not been populated
[INFO]: Time taken for simulation start : 1.887725 seconds
[INFO] Action Manager:  <ActionManager> contains 4 active terms.
+---------------------------------------------+
|       Active Action Terms (shape: 16)       |
+-------+-------------------------+-----------+
| Index | Name                    | Dimension |
+-------+-------------------------+-----------+
|   0   | robot1_joint_torques    |         7 |
|   1   | robot1_finger_joint_pos |         1 |
|   2   | robot2_joint_torques    |         7 |
|   3   | robot2_finger_joint_pos |         1 |
+-------+-------------------------+-----------+

[INFO] Observation Manager: <ObservationManager> contains 1 groups.
+-------------------------------------------------------------+
| Active Observation Terms in Group: 'LowLevelActions' (shape: (36,)) |
+---------------+-----------------------------+---------------+
|     Index     | Name                        |     Shape     |
+---------------+-----------------------------+---------------+
|       0       | joint_pos_1                 |      (9,)     |
|       1       | joint_vel_1                 |      (9,)     |
|       2       | joint_pos_2                 |      (9,)     |
|       3       | joint_vel_2                 |      (9,)     |
+---------------+-----------------------------+---------------+

[INFO] Event Manager:  <EventManager> contains 1 active terms.
+---------------------------------------+
|  Active Event Terms in Mode: 'reset'  |
+--------+------------------------------+
| Index  | Name                         |
+--------+------------------------------+
|   0    | reset_all                    |
|   1    | reset_object_position        |
|   2    | reset_robot_joint_position   |
+--------+------------------------------+

Creating window for environment.
Se3SpaceMouse Controller
    Mode: translation
    Position Sensitivity: 0.75
    Rotation Sensitivity: 1.2

[659.976s] Simulation App Shutting Down

I posted it as a question as I'm not sure if it is a bug or something related to the migration. I followed the migration guide and did the necessary changes

EDIT: I tried to load the robot and bin through the gui and both are loaded well: load

jtigue-bdai commented 5 months ago

For your robot2, it may be having issues with references to the mesh assets in : "/home/mikel/IsaacOrbitLFD/IsaacLab/source/extensions/omni.isaac.lab_assets/data/Robots/franka/panda.usd".

Is that usd an imported URDF? If so you can do a direct import in the scene config using the UrdfFileCfg instead of the UsDFileCfg.

mikelasa commented 5 months ago

Thank you for answering, actually I found the issue. The urdf was importing well, at some point I changed the initial position and rotation, were I left rotation in (0, 0, 0, 0) causing to vanish the robot and the box.

jbt-cmu commented 5 months ago

For your robot2, it may be having issues with references to the mesh assets in : "/home/mikel/IsaacOrbitLFD/IsaacLab/source/extensions/omni.isaac.lab_assets/data/Robots/franka/panda.usd".

Is that usd an imported URDF? If so you can do a direct import in the scene config using the UrdfFileCfg instead of the UsDFileCfg.

I have noticed that if you load any URDF with Isaac Lab, the RGB information is terribly wrong (its typically all white). The same happens with USDView in Omniverse, even when loading Nvidia's Carter and exporting to USD.

jtigue-bdai commented 5 months ago

How is your URDF specifying RGB information? Are you utilizing .dae (collada files) or the route?

jbt-cmu commented 5 months ago

My URDF, the Hello Robot stretch, does it like so:

<robot name="stretch">
  <link name="base_link">
    <inertial>
      <origin rpy="0 0 0" xyz="-0.087526 -0.001626 0.081009"/>
      <mass value="17.384389"/>
      <inertia ixx="0.160002" ixy="0.006758" ixz="0.004621" iyy="0.138068" iyz="0.002208" izz="0.228992"/>
    </inertial>
    <visual>
      <origin rpy="0 0 0" xyz="0 0 0"/>
      <geometry>
        <mesh filename="./meshes/base_link.STL"/>
      </geometry>
      <material name="">
        <color rgba="0.79216 0.81961 0.93333 1"/>
      </material>
    </visual>
    <collision>
      <origin rpy="0 0 0" xyz="0 0 0"/>
      <geometry>
        <mesh filename="./meshes/base_link_collision.STL"/>
      </geometry>
    </collision>
  </link>
Mayankm96 commented 5 months ago

@mikelasa All zeros is an invalid quaternion. Can you fix this and see if that solves your problem?

    robot2.init_state.rot = (0.0, 0.0, 0.0, 0.0)

@jbt-cmu Regarding textures, let's discuss that in the issues you raised. It's very likely that you need to post this issue on the Isaac Sim forums. The issue seems to be with URDF importer from Isaac Sim and not Isaac Lab.

mikelasa commented 5 months ago

@Mayankm96 Yes, I already commented above that this was the issue, at some point I changed that to 0 the robot wasn't spawning, thank you