search

NVlabs / curobo

CUDA Accelerated Robot Library
https://curobo.org
Other
722 stars 107 forks source link

Unable to load NVblox map and errors about urdf assets being broken in isaac_sim docker #339

Open robertisaac-rgb opened 1 month ago

robertisaac-rgb commented 1 month ago

System info

  1. cuRobo installation mode: docker isaac sim
  2. python version: Python 3.10.13
  3. Isaac Sim version: 2023.1.0

Issue Details: I am following the updated instructions here for Install for use in Isaac Sim

When I run omni_python examples/isaac_sim/motion_gen_reacher_nvblox.py

I get the following.

[15.673s] Simulation App Startup Complete
2024-07-17 00:49:14 [17,505ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,505ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link0/visuals not created
2024-07-17 00:49:14 [17,523ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,523ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link1/visuals not created
2024-07-17 00:49:14 [17,542ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,542ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link2/visuals not created
2024-07-17 00:49:14 [17,555ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,555ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link3/visuals not created
2024-07-17 00:49:14 [17,567ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,567ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link4/visuals not created
2024-07-17 00:49:14 [17,587ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,587ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link5/visuals not created
2024-07-17 00:49:14 [17,601ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,601ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link6/visuals not created
2024-07-17 00:49:14 [17,613ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,613ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_link7/visuals not created
2024-07-17 00:49:14 [17,629ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,629ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_hand/visuals not created
2024-07-17 00:49:14 [17,643ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,643ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_leftfinger/visuals not created
2024-07-17 00:49:14 [17,659ms] [Warning] [omni.importer.urdf] Asset convert failed as asset file is broken.
2024-07-17 00:49:14 [17,659ms] [Warning] [omni.importer.urdf] Prim /World/panda/panda_rightfinger/visuals not created
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240716 17:49:15.461912   426 layer_cake_impl.h:32] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_9TsdfVoxelEEE, voxel_size: 0.03, and memory_type: kDevice to LayerCake.
I20240716 17:49:15.461973   426 layer_cake_impl.h:32] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_10ColorVoxelEEE, voxel_size: 0.03, and memory_type: kDevice to LayerCake.
I20240716 17:49:15.461988   426 layer_cake_impl.h:32] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_14FreespaceVoxelEEE, voxel_size: 0.03, and memory_type: kDevice to LayerCake.
I20240716 17:49:15.462002   426 layer_cake_impl.h:32] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_14OccupancyVoxelEEE, voxel_size: 0.03, and memory_type: kDevice to LayerCake.
I20240716 17:49:15.462014   426 layer_cake_impl.h:32] Adding Layer with type: N6nvblox15VoxelBlockLayerINS_9EsdfVoxelEEE, voxel_size: 0.03, and memory_type: kDevice to LayerCake.
I20240716 17:49:15.462028   426 layer_cake_impl.h:32] Adding Layer with type: N6nvblox10BlockLayerINS_9MeshBlockEEE, voxel_size: 0.03, and memory_type: kDevice to LayerCake.
E20240716 17:49:15.462833   426 sqlite_database.cpp:192] Preparing query failed: file is not a database
E20240716 17:49:15.462893   426 mapper.cpp:536] Failed to load map from file: /isaac-sim/kit/python/lib/python3.10/site-packages/curobo/content/assets/scene/nvblox/srl_ur10_bins.nvblx
warming up...
Curobo is Ready
[33.985s] [ext: omni.kit.asset_converter-2.0.12] startup
[34.022s] [ext: omni.kit.tool.asset_importer-2.4.18] startup
[34.103s] [ext: omni.isaac.asset_browser-0.3.1] startup
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_rightfinger/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_leftfinger/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_hand/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link7/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link5/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link6/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link3/collisions
2024-07-17 00:49:31 [34,219ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link4/collisions
2024-07-17 00:49:31 [34,220ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link2/collisions
2024-07-17 00:49:31 [34,220ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link1/collisions
2024-07-17 00:49:31 [34,220ms] [Error] [omni.physicsschema.plugin] Provided mesh geom with a PhysicsCollisionAPI does not have points, collision will not be created. Prim: /World/panda/panda_link0/collisions
Traceback (most recent call last):
  File "/pkgs/curobo/examples/isaac_sim/motion_gen_reacher_nvblox.py", line 294, in <module>
    main()
  File "/pkgs/curobo/examples/isaac_sim/motion_gen_reacher_nvblox.py", line 166, in main
    usd_help.add_world_to_stage(world_cfg.get_mesh_world(), base_frame="/World")
  File "/isaac-sim/kit/python/lib/python3.10/site-packages/curobo/util/usd_helper.py", line 539, in add_world_to_stage
    prim_path = [
  File "/isaac-sim/kit/python/lib/python3.10/site-packages/curobo/util/usd_helper.py", line 540, in <listcomp>
    self.get_prim_from_obstacle(o, full_path, timestep=timestep) for o in obstacles.objects
  File "/isaac-sim/kit/python/lib/python3.10/site-packages/curobo/util/usd_helper.py", line 550, in get_prim_from_obstacle
    return self.add_mesh_to_stage(obstacle, base_frame, timestep=timestep)
  File "/isaac-sim/kit/python/lib/python3.10/site-packages/curobo/util/usd_helper.py", line 636, in add_mesh_to_stage
    set_geom_mesh_attrs(obj_geom, obstacle, timestep=timestep)
  File "/isaac-sim/kit/python/lib/python3.10/site-packages/curobo/util/usd_helper.py", line 162, in set_geom_mesh_attrs
    if max(np.ravel(obs.vertex_colors) > 1.0):
ValueError: max() arg is an empty sequence
2024-07-17 00:49:31 [34,429ms] [Warning] [omni.core.ITypeFactory] Module /isaac-sim/kit/exts/omni.graph.action/bin/libomni.graph.action.plugin.so remained loaded after unload request.
2024-07-17 00:49:31 [34,435ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /isaac-sim/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2024-07-17 00:49:31 [34,444ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2024-07-17 00:49:31 [34,448ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,448ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,448ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,448ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,449ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-07-17 00:49:31 [34,457ms] [Warning] [carb.audio.context] 1 contexts were leaked
2024-07-17 00:49:31 [34,575ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2024-07-17 00:49:31 [34,596ms] [Warning] [omni.core.ITypeFactory] Module /isaac-sim/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.

After I get to this point my Isaac Sim window closes and I am unable to see any output. Any ideas what is going on and how I can fix it?

balakumar-s commented 1 month ago

Can you try the latest main? We fixed the max(np.ravel(obs.vertex_colors) > 1.0): error you were seeing.