search

leggedrobotics / viplanner

ViPlanner: Visual Semantic Imperative Learning for Local Navigation
https://leggedrobotics.github.io/viplanner.github.io/
Other
341 stars 35 forks source link

[Bug Report] Bug title ./isaaclab.sh -p <path-to-viplanner-repo>/omniverse/standalone/viplanner_demo.py find error #44

Open zhw970623 opened 6 days ago

zhw970623 commented 6 days ago

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug

A clear and concise description of what the bug is.

When I verify the model demo, I execute cd IsaacLab ./isaaclab.sh -p /omniverse/standalone/viplanner_demo.py --scene carla --model_dir and get an error

Steps to reproduce

Please try to provide a minimal example to reproduce the bug. Error messages and stack traces are also helpful.

System Info

Describe the characteristic of your environment:

pascal-roth commented 5 days ago

Hi,

Thanks for using our work. Please include the error message as otherwise there is no way how do help from our side.

zhw970623 commented 5 days ago

viplanner 0.1.0 requires wandb==0.14.0,but you project need rl-games 1.6.1 , and rl-games 1.6.1 need requires wandb<0.13.0,>=0.12.11 error message: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. rl-games 1.6.1 requires wandb<0.13.0,>=0.12.11, but you have wandb 0.14.0 which is incompatible.

RROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. viplanner 0.1.0 requires wandb==0.14.0, but you have wandb 0.12.11 which is incompatible.

pascal-roth commented 5 days ago

These pip errors should not influence the system if it normally installs. Did you try to run the demo?

zhw970623 commented 5 days ago

Yes, I tried the demo, and I allowed this command ./isaaclab.sh -p /omniverse/standalone/viplanner_demo.py --scene carla --model_dir ex:./isaaclab.sh -p /home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py --scene carla --model_dir /home/zhw/reach/baselineViplanner/modle/model.pt

And I put usd_path="${USER_PATH_TO_USD}/carla.usd ---> in the file /home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.viplanner/omni/viplanner/config/carla_cfg.py usd_path="/home/zhw/reach/baselineViplanner/modle/new_carla_export/carla.usd"

Then the crash occurs, and the final error is as follows

`Traceback (most recent call last):
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 64, in <lambda>
    lambda event, obj=weakref.proxy(self): obj._initialize_callback(event),
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 271, in _initialize_callback
    self._initialize_impl()
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py", line 388, in _initialize_impl
    import omni.replicator.core as rep
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/__init__.py", line 3, in <module>
    from .scripts import (
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/__init__.py", line 3, in <module>
    from . import (
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/__init__.py", line 20, in <module>
    from .s3 import S3Backend
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/s3.py", line 19, in <module>
    import boto3
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/__init__.py", line 16, in <module>
    from boto3.compat import _warn_deprecated_python
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/compat.py", line 19, in <module>
    from boto3.exceptions import PythonDeprecationWarning
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/exceptions.py", line 15, in <module>
    import botocore.exceptions
ModuleNotFoundError: No module named 'botocore'
Traceback (most recent call last):
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 64, in <lambda>
    lambda event, obj=weakref.proxy(self): obj._initialize_callback(event),
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 271, in _initialize_callback
    self._initialize_impl()
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py", line 388, in _initialize_impl
    import omni.replicator.core as rep
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/__init__.py", line 3, in <module>
    from .scripts import (
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/__init__.py", line 3, in <module>
    from . import (
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/__init__.py", line 20, in <module>
    from .s3 import S3Backend
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/s3.py", line 19, in <module>
    import boto3
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/__init__.py", line 16, in <module>
    from boto3.compat import _warn_deprecated_python
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/compat.py", line 19, in <module>
    from boto3.exceptions import PythonDeprecationWarning
  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/exceptions.py", line 15, in <module>
    import botocore.exceptions
ModuleNotFoundError: No module named 'botocore'
2024-11-22 09:05:22 [24,876ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /Visuals/RayCaster.proto0_mesh_id0 that has not been populated
2024-11-22 09:05:22 [25,064ms] [Warning] [omni.hydra] Mesh '/World/envs/env_0/Robot/base/visuals.proto_mesh_0_id0' has corrupted data in primvar 'st': buffer size 702 doesn't match expected size 12828 in faceVarying primvars
[INFO]: Time taken for simulation start : 13.749385 seconds
[INFO] Command Manager:  <CommandManager> contains 1 active terms.
+----------------------------------------------------+
|                Active Command Terms                |
+-------+-------------+------------------------------+
| Index | Name        |             Type             |
+-------+-------------+------------------------------+
|   0   | vel_command | PathFollowerCommandGenerator |
+-------+-------------+------------------------------+

[INFO] Action Manager:  <ActionManager> contains 1 active terms.
+----------------------------------+
| Active Action Terms (shape: 153) |
+---------+---------+--------------+
|  Index  | Name    |    Dimension |
+---------+---------+--------------+
|    0    | paths   |          153 |
+---------+---------+--------------+

Module omni.isaac.lab.utils.warp.kernels 6cb40f6 load on device 'cuda:0' took 0.33 ms
Traceback (most recent call last):
  File "/home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py", line 151, in <module>
    main()
  File "/home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py", line 71, in main
    env = ManagerBasedRLEnv(env_cfg)
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_rl_env.py", line 75, in __init__
    super().__init__(cfg=cfg)
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_env.py", line 136, in __init__
    self.load_managers()
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_rl_env.py", line 115, in load_managers
    super().load_managers()
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_env.py", line 213, in load_managers
    self.observation_manager = ObservationManager(self.cfg.observations, self)
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/managers/observation_manager.py", line 69, in __init__
    super().__init__(cfg, env)
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/managers/manager_base.py", line 130, in __init__
    self._prepare_terms()
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/managers/observation_manager.py", line 337, in _prepare_terms
    obs_dims = tuple(term_cfg.func(self._env, **term_cfg.params).shape)
  File "/home/zhw/reach/baselineViplanner/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/viplanner/mdp/observations.py", line 51, in isaac_camera_data
    sensor: CameraData = env.scene.sensors[sensor_cfg.name].data
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py", line 174, in data
    self._update_outdated_buffers()
  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 284, in _update_outdated_buffers
    outdated_env_ids = self._is_outdated.nonzero().squeeze(-1)
AttributeError: 'Camera' object has no attribute '_is_outdated'
2024-11-22 09:05:26 [28,836ms] [Warning] [carb] Recursive unloadAllPlugins() detected!`
zhw970623 commented 5 days ago

I also downloaded the model.yaml file, but I don't know which directory this file should be placed in，but When I execute /isaaclab.sh -p /home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py --scene carla --model_dir /home/zhw/reach/baselineViplanner/modle/model.ptcommand, model.pt and modle.yaml are in the same folder

pascal-roth commented 4 days ago

I also downloaded the model.yaml file, but I don't know which directory this file should be placed in，but

When I execute /isaaclab.sh -p /home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py --scene carla --model_dir /home/zhw/reach/baselineViplanner/modle/model.ptcommand, model.pt and modle.yaml are in the same folder

The '--model_dir' should be '/home/zhw/reach/baselineViplanner/modle'. Then both model.pt and model.yaml should be in there

pascal-roth commented 4 days ago

Yes, I tried the demo, and I allowed this command

./isaaclab.sh -p /omniverse/standalone/viplanner_demo.py --scene carla --model_dir

ex:./isaaclab.sh -p /home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py --scene carla --model_dir /home/zhw/reach/baselineViplanner/modle/model.pt

And I put usd_path="${USER_PATH_TO_USD}/carla.usd ---> in the file /home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.viplanner/omni/viplanner/config/carla_cfg.py usd_path="/home/zhw/reach/baselineViplanner/modle/new_carla_export/carla.usd"

Then the crash occurs, and the final error is as follows


`Traceback (most recent call last):

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 64, in <lambda>

    lambda event, obj=weakref.proxy(self): obj._initialize_callback(event),

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 271, in _initialize_callback

    self._initialize_impl()

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py", line 388, in _initialize_impl

    import omni.replicator.core as rep

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/__init__.py", line 3, in <module>

    from .scripts import (

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/__init__.py", line 3, in <module>

    from . import (

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/__init__.py", line 20, in <module>

    from .s3 import S3Backend

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/s3.py", line 19, in <module>

    import boto3

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/__init__.py", line 16, in <module>

    from boto3.compat import _warn_deprecated_python

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/compat.py", line 19, in <module>

    from boto3.exceptions import PythonDeprecationWarning

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/exceptions.py", line 15, in <module>

    import botocore.exceptions

ModuleNotFoundError: No module named 'botocore'

Traceback (most recent call last):

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 64, in <lambda>

    lambda event, obj=weakref.proxy(self): obj._initialize_callback(event),

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 271, in _initialize_callback

    self._initialize_impl()

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py", line 388, in _initialize_impl

    import omni.replicator.core as rep

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/__init__.py", line 3, in <module>

    from .scripts import (

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/__init__.py", line 3, in <module>

    from . import (

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/__init__.py", line 20, in <module>

    from .s3 import S3Backend

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/extscache/omni.replicator.core-1.11.20+106.1.0.lx64.r.cp310/omni/replicator/core/scripts/backends/s3.py", line 19, in <module>

    import boto3

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/__init__.py", line 16, in <module>

    from boto3.compat import _warn_deprecated_python

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/compat.py", line 19, in <module>

    from boto3.exceptions import PythonDeprecationWarning

  File "/home/zhw/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.pip.cloud/pip_prebundle/boto3/exceptions.py", line 15, in <module>

    import botocore.exceptions

ModuleNotFoundError: No module named 'botocore'

2024-11-22 09:05:22 [24,876ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /Visuals/RayCaster.proto0_mesh_id0 that has not been populated

2024-11-22 09:05:22 [25,064ms] [Warning] [omni.hydra] Mesh '/World/envs/env_0/Robot/base/visuals.proto_mesh_0_id0' has corrupted data in primvar 'st': buffer size 702 doesn't match expected size 12828 in faceVarying primvars

[INFO]: Time taken for simulation start : 13.749385 seconds

[INFO] Command Manager:  <CommandManager> contains 1 active terms.

+----------------------------------------------------+

|                Active Command Terms                |

+-------+-------------+------------------------------+

| Index | Name        |             Type             |

+-------+-------------+------------------------------+

|   0   | vel_command | PathFollowerCommandGenerator |

+-------+-------------+------------------------------+

[INFO] Action Manager:  <ActionManager> contains 1 active terms.

+----------------------------------+

| Active Action Terms (shape: 153) |

+---------+---------+--------------+

|  Index  | Name    |    Dimension |

+---------+---------+--------------+

|    0    | paths   |          153 |

+---------+---------+--------------+

Module omni.isaac.lab.utils.warp.kernels 6cb40f6 load on device 'cuda:0' took 0.33 ms

Traceback (most recent call last):

  File "/home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py", line 151, in <module>

    main()

  File "/home/zhw/reach/baselineViplanner/viplanner/omniverse/standalone/viplanner_demo.py", line 71, in main

    env = ManagerBasedRLEnv(env_cfg)

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_rl_env.py", line 75, in __init__

    super().__init__(cfg=cfg)

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_env.py", line 136, in __init__

    self.load_managers()

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_rl_env.py", line 115, in load_managers

    super().load_managers()

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/envs/manager_based_env.py", line 213, in load_managers

    self.observation_manager = ObservationManager(self.cfg.observations, self)

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/managers/observation_manager.py", line 69, in __init__

    super().__init__(cfg, env)

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/managers/manager_base.py", line 130, in __init__

    self._prepare_terms()

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/managers/observation_manager.py", line 337, in _prepare_terms

    obs_dims = tuple(term_cfg.func(self._env, **term_cfg.params).shape)

  File "/home/zhw/reach/baselineViplanner/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/viplanner/mdp/observations.py", line 51, in isaac_camera_data

    sensor: CameraData = env.scene.sensors[sensor_cfg.name].data

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py", line 174, in data

    self._update_outdated_buffers()

  File "/home/zhw/reach/baselineViplanner/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/sensor_base.py", line 284, in _update_outdated_buffers

    outdated_env_ids = self._is_outdated.nonzero().squeeze(-1)

AttributeError: 'Camera' object has no attribute '_is_outdated'

2024-11-22 09:05:26 [28,836ms] [Warning] [carb] Recursive unloadAllPlugins() detected!`

This looks to me like an error in your IsaacLab installation as replicator fails which has nothing to do with this code. Please redirect the issue there.

fan-ziqi commented 3 days ago

Have you checkout the specified commit in the README?

IsaacLab 1.2.0 (commit: 0bccd886c19a9891c0b6bdd37e13f338eacc0bba)