search

leggedrobotics / viplanner

ViPlanner: Visual Semantic Imperative Learning for Local Navigation
https://leggedrobotics.github.io/viplanner.github.io/
Other
307 stars 31 forks source link

semantic map information #12

Closed Ruihyw closed 4 months ago

Ruihyw commented 5 months ago

Hi!, thank you for the great work of your team！ I have successfully run the demo.py and I was wondering about how to view the semantic map information in the demo program. Looking forward for your reply！

pascal-roth commented 5 months ago

Hi, thanks a lot for using our repo.

We are currently working on a new extension that facilitates that, will keep you updated. In the meantime, at least for the Carla environment, you can visualise the semantics of the camera using the standard Isaac way https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator/visualization.html#visualization

Please let us know if there are any other questions or issues.

Ruihyw commented 5 months ago

Thank you very much for your reply, I managed to get the semantic information now!

But actually I can't actually run it with carla data , so I'd appreciate it if you could help me with that! here is the warning message: 2024-05-24 06:09:17 [19,013ms] [Warning] [omni.viplanner.utils.unreal_importer] Mesh Town01_Opt has no faces or vertices. Execute crosswalk multiplication 'cw_2' 2024-05-24 06:09:17 [19,367ms] [Warning] [omni.usd._impl.utils] Cannot duplicate prim /World/Carla/terrain/Town01_Opt/Town01_Opt/Road_Crosswalk_Town01_2 since it doesn't exist. [INFO]: Time taken for scene creation : 0.915456 seconds Traceback (most recent call last): File "/home/wwd/hyw/viplanner/omniverse/standalone/viplanner_demo.py", line 135, in main() File "/home/wwd/hyw/viplanner/omniverse/standalone/viplanner_demo.py", line 71, in main env = RLTaskEnv(env_cfg) File "/home/wwd/hyw/orbit/source/extensions/omni.isaac.orbit/omni/isaac/orbit/envs/rl_task_env.py", line 84, in init super().init(cfg=cfg) File "/home/wwd/hyw/orbit/source/extensions/omni.isaac.orbit/omni/isaac/orbit/envs/base_env.py", line 119, in init self.scene = InteractiveScene(self.cfg.scene) File "/home/wwd/hyw/orbit/source/extensions/omni.isaac.orbit/omni/isaac/orbit/scene/interactive_scene.py", line 142, in init self._add_entities_from_cfg() File "/home/wwd/hyw/orbit/source/extensions/omni.isaac.orbit/omni/isaac/orbit/scene/interactive_scene.py", line 350, in _add_entities_from_cfg self.terrain = asset_cfg.class_type(asset_cfg) File "/home/wwd/hyw/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/utils/unreal_importer.py", line 44, in init self._multiply_crosswalks() File "/home/wwd/hyw/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/utils/unreal_importer.py", line 236, in _multiply_crosswalks assert success, f"Failed to duplicate crosswalk '{key}'" AssertionError: Failed to duplicate crosswalk 'cw_2' 2024-05-24 06:09:17 [19,379ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/wwd/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request 2024-05-24 06:09:17 [19,381ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead. 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-05-24 06:09:17 [19,383ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal 2024-05-24 06:09:17 [19,386ms] [Warning] [carb.audio.context] 1 contexts were leaked 2024-05-24 06:09:17 [19,433ms] [Warning] [carb] Recursive unloadAllPlugins() detected! 2024-05-24 06:09:17 [19,448ms] [Warning] [omni.core.ITypeFactory] Module /home/wwd/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.

pascal-roth commented 5 months ago

Hi, this looks like an error in the mesh file. I will look into it

pascal-roth commented 4 months ago

The mesh has been updated and should work again as expected. Please redownload the asset and also pull the latest changes of the repo.

Closing this issue for now, please reopen if there are any further issues on your side.