search

facebookresearch / habitat-sim

A flexible, high-performance 3D simulator for Embodied AI research.
https://aihabitat.org/
MIT License
2.55k stars 414 forks source link

Replica-Dataset + Bullet Physics crashes #613

Open soumith opened 4 years ago

soumith commented 4 years ago

🐛 Bug

Any environment from https://github.com/facebookresearch/Replica-Dataset fails to work when enabling Bullet physics.

It works with:

The exact spot in the logs that fails is:

E0502 04:55:08.508623   972 BulletPhysicsManager.cpp:84] Invalid primitive: Points
E0502 04:55:08.508630   972 BulletPhysicsManager.cpp:101] Cannot load collision mesh, skipping

Sometimes it's Invalid primitive: Points, sometimes BulletPhysicsManager.cpp:99] Invalid primitive -2101531424, etc. It looks like some kind of overflow or stack corruption from a quick glance.

Command

The paths I use:

Logs

==== Initialized Sensor Spec: =====
Sensor uuid:  rgb
Sensor type:  SensorType.COLOR
Sensor position:  [0.  0.6 0. ]
===================================
==== Initialized Sensor Spec: =====
Sensor uuid:  depth
Sensor type:  SensorType.DEPTH
Sensor position:  [0.  0.6 0. ]
===================================
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0502 04:48:56.452706   905 Simulator.cpp:64] Loading navmesh from /remote/Replica-Dataset/data/apartment_0/habitat/mesh_semantic.navmesh
I0502 04:48:56.452836   905 Simulator.cpp:66] Loaded.
I0502 04:48:56.452862   905 SceneGraph.h:92] Created DrawableGroup:
I0502 04:48:56.457373   905 WindowlessContext.cpp:100] [EGL] Detected 3 EGL devices
I0502 04:48:56.476143   905 WindowlessContext.cpp:121] [EGL] Selected EGL device 0 for CUDA device 0
I0502 04:48:56.478258   905 WindowlessContext.cpp:135] [EGL] Version: 1.5
I0502 04:48:56.478272   905 WindowlessContext.cpp:136] [EGL] Vendor: NVIDIA
Renderer: GeForce GTX TITAN X/PCIe/SSE2 by NVIDIA Corporation
OpenGL version: 4.6.0 NVIDIA 435.21
Using optional features:
    GL_ARB_ES2_compatibility
    GL_ARB_direct_state_access
    GL_ARB_get_texture_sub_image
    GL_ARB_invalidate_subdata
    GL_ARB_multi_bind
    GL_ARB_robustness
    GL_ARB_separate_shader_objects
    GL_ARB_texture_filter_anisotropic
    GL_ARB_texture_storage
    GL_ARB_texture_storage_multisample
    GL_ARB_vertex_array_object
    GL_KHR_debug
Using driver workarounds:
    nv-egl-incorrect-gl11-function-pointers
    no-layout-qualifiers-on-old-glsl
    nv-zero-context-profile-mask
    nv-implementation-color-read-format-dsa-broken
    nv-cubemap-inconsistent-compressed-image-size
    nv-cubemap-broken-full-compressed-image-query
    nv-compressed-block-size-in-bits
W0502 04:48:56.491065   905 ResourceManager.cpp:357] Cannot find /remote/habitat-sim/data/objects/cheezit or /remote/habitat-sim/data/objects/cheezit.phys_properties.json. Aborting parse.
W0502 04:48:56.491075   905 ResourceManager.cpp:357] Cannot find /remote/habitat-sim/data/objects/chefcan or /remote/habitat-sim/data/objects/chefcan.phys_properties.json. Aborting parse.
W0502 04:48:56.491082   905 ResourceManager.cpp:357] Cannot find /remote/habitat-sim/data/objects/banana or /remote/habitat-sim/data/objects/banana.phys_properties.json. Aborting parse.
I0502 04:49:53.353385   938 ResourceManager.cpp:182] loaded object templates: 0
E0502 04:49:53.353519   938 BulletPhysicsManager.cpp:99] Invalid primitive 11
E0502 04:49:53.353525   938 BulletPhysicsManager.cpp:101] Cannot load collision mesh, skipping
E0502 04:49:53.353529   938 Simulator.cpp:124] cannot load /remote/Replica-Dataset/data/apartment_0/habitat/mesh_semantic.ply
I0502 04:49:53.353652   938 BulletPhysicsManager.cpp:16] Deconstructing BulletPhysicsManager
I0502 04:49:53.353688   938 PhysicsManager.cpp:32] Deconstructing PhysicsManager
I0502 04:49:53.353693   938 SceneManager.h:24] Deconstructing SceneManager
I0502 04:49:53.353698   938 SceneGraph.h:25] Deconstructing SceneGraph
I0502 04:49:53.357825   938 Renderer.cpp:33] Deconstructing Renderer
I0502 04:49:53.357841   938 WindowlessContext.h:16] Deconstructing WindowlessContext
I0502 04:49:53.357843   938 WindowlessContext.cpp:245] Deconstructing GL context
Traceback (most recent call last):
  File "remote_locobot.py", line 309, in <module>
    robot = RemoteLoCoRobot(backend=args.backend, backend_config=args.backend_config)
  File "remote_locobot.py", line 30, in __init__
    self._robot = Robot(backend, common_config=backend_config)
  File "/root/low_cost_ws/src/pyrobot/src/pyrobot/core.py", line 120, in __init__
    common_class(self.configs, **common_config),
  File "/root/low_cost_ws/src/pyrobot/src/pyrobot/habitat/simulator.py", line 24, in __init__
    self.sim = habitat_sim.Simulator(make_cfg(self.sim_config))
  File "<attrs generated init habitat_sim.simulator.Simulator>", line 9, in __init__
  File "/root/pyenv_pyrobot_python3/lib/python3.6/site-packages/habitat_sim-0.1.4-py3.6-linux-x86_64.egg/habitat_sim/simulator.py", line 67, in __attrs_post_init__
    self.reconfigure(config)
  File "/root/pyenv_pyrobot_python3/lib/python3.6/site-packages/habitat_sim-0.1.4-py3.6-linux-x86_64.egg/habitat_sim/simulator.py", line 193, in reconfigure
    self._config_backend(config)
  File "/root/pyenv_pyrobot_python3/lib/python3.6/site-packages/habitat_sim-0.1.4-py3.6-linux-x86_64.egg/habitat_sim/simulator.py", line 111, in _config_backend
    self._sim = SimulatorBackend(config.sim_cfg)
ValueError: Cannot load: /remote/Replica-Dataset/data/apartment_0/habitat/mesh_semantic.ply

Expected behavior

The scene loads without crash

erikwijmans commented 4 years ago

We likely need to decimate replica meshes for them to work with physics. We should (if possible) fix whatever bug is causing this but I would be surprised if Bullet was useable with replica meshes in their current form (I think the one you are loading has ~14 million vertices). We had started working on adding a mesh decimation tool here: https://github.com/facebookresearch/habitat-sim/pull/468

giuliano-97 commented 2 years ago

We likely need to decimate replica meshes for them to work with physics. We should (if possible) fix whatever bug is causing this but I would be surprised if Bullet was useable with replica meshes in their current form (I think the one you are loading has ~14 million vertices). We had started working on adding a mesh decimation tool here: #468

@erikwijmans I tried to decimate the room_2 semantic mesh to <500000 vertices but it still doesn't work (roughly same error as OP):

I0523 17:02:43.036067 734671 Simulator.cpp:206] ::setSceneInstanceAttributes : Navmesh file location in scene instance : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.navmesh
I0523 17:02:43.036077 734671 Simulator.cpp:213] ::setSceneInstanceAttributes : Loading navmesh from /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.navmesh
I0523 17:02:43.036129 734671 Simulator.cpp:216] ::setSceneInstanceAttributes : Navmesh Loaded.
I0523 17:02:43.036149 734671 SceneGraph.h:85] Created DrawableGroup: 
I0523 17:02:43.036154 734671 Simulator.cpp:251] ::setSceneInstanceAttributes : SceneInstance : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply proposed Semantic Scene Descriptor filename : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.scn
E0523 17:02:43.036163 734671 SemanticScene.h:155] ::loadSemanticSceneDescriptor : File /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.scn does not exist.  Aborting load.
W0523 17:02:43.036460 734671 Simulator.cpp:275] ::setSceneInstanceAttributes : All attempts to load SSD with SceneAttributes-provided name /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.scn : exist : 0 : loaded as expected type : 1
I0523 17:02:43.109735 734671 Simulator.cpp:324] ::createSceneInstance : Using scene instance-specified Light key : -no_lights-
I0523 17:02:43.109759 734671 MetadataMediator.cpp:68] ::setSimulatorConfiguration : Set new simulator config for scene/stage : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply and dataset : default which is currently active dataset.
I0523 17:02:43.109795 734671 Simulator.cpp:382] ::createSceneInstance : Start to load stage named : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply with render asset : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply and collision asset : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply
W0523 17:02:43.109840 734671 ResourceManager.cpp:252] ::loadStage : Not loading semantic mesh - File Name : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic_semantic.ply does not exist.
I0523 17:02:43.109850 734671 ResourceManager.cpp:285] ::loadStage : start load render asset /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply.
I0523 17:02:43.109856 734671 ResourceManager.cpp:699] ::loadStageInternal : Attempting to load stage /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply 
E0523 17:02:43.909266 734671 BulletPhysicsManager.cpp:265] Invalid primitive 0
E0523 17:02:43.909301 734671 BulletPhysicsManager.cpp:267] Cannot load collision mesh, skipping
E0523 17:02:43.909307 734671 ResourceManager.cpp:339] ::loadStage : Adding Stage /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply to PhysicsManager failed. Aborting scene initialization.
E0523 17:02:43.909317 734671 Simulator.cpp:394] ::createSceneInstance : Cannot load stage : /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply
I0523 17:02:43.911481 734671 BulletPhysicsManager.cpp:35] Deconstructing BulletPhysicsManager
I0523 17:02:43.911545 734671 PhysicsManager.cpp:50] Deconstructing PhysicsManager
I0523 17:02:43.911561 734671 SemanticScene.h:43] Deconstructing SemanticScene
I0523 17:02:43.911584 734671 SceneManager.h:25] Deconstructing SceneManager
I0523 17:02:43.911589 734671 SceneGraph.h:25] Deconstructing SceneGraph
I0523 17:02:43.912503 734671 Renderer.cpp:71] Deconstructing Renderer
I0523 17:02:43.912528 734671 WindowlessContext.h:17] Deconstructing WindowlessContext
...
    self.sim = habitat_sim.Simulator(self.cfg)
  File "<attrs generated init habitat_sim.simulator.Simulator>", line 11, in __init__
  File "/home/giuliano/miniconda3/envs/robostackenv/lib/python3.8/site-packages/habitat_sim/simulator.py", line 121, in __attrs_post_init__
    self.__set_from_config(self.config)
  File "/home/giuliano/miniconda3/envs/robostackenv/lib/python3.8/site-packages/habitat_sim/simulator.py", line 252, in __set_from_config
    self._config_backend(config)
  File "/home/giuliano/miniconda3/envs/robostackenv/lib/python3.8/site-packages/habitat_sim/simulator.py", line 196, in _config_backend
    super().__init__(config.sim_cfg, config.metadata_mediator)
ValueError: ::createSceneInstance : Cannot load: /media/giuliano/wd_ssd/datasets/replica/room_2/habitat/mesh_semantic.ply

were you able to figure out what is causing this?