search

facebookresearch / habitat-challenge

Code for the habitat challenge
https://aihabitat.org
MIT License
308 stars 56 forks source link

Error Evaluate your docker container locally #112

Closed ricard-inho closed 2 years ago

ricard-inho commented 2 years ago

I'm on the last step of setting everything in my docker container and test it out. But when I run

./test_locally_objectnav_rgbd.sh --docker-name objectnav_submission

I get the following error:

Traceback (most recent call last):
  File "agent.py", line 41, in <module>
    main()
  File "agent.py", line 33, in main
    challenge = habitat.Challenge(eval_remote=False)
  File "/habitat-lab/habitat/core/challenge.py", line 16, in __init__
    super().__init__(config_paths, eval_remote=eval_remote)
  File "/habitat-lab/habitat/core/benchmark.py", line 38, in __init__
    self._env = Env(config=config_env)
  File "/habitat-lab/habitat/core/env.py", line 108, in __init__
    id_sim=self._config.SIMULATOR.TYPE, config=self._config.SIMULATOR
  File "/habitat-lab/habitat/sims/registration.py", line 19, in make_sim
    return _sim(**kwargs)
  File "/habitat-lab/habitat/sims/habitat_simulator/habitat_simulator.py", line 271, in __init__
    super().__init__(self.sim_config)
  File "<attrs generated init habitat_sim.simulator.Simulator>", line 11, in __init__
  File "/opt/conda/envs/habitat/lib/python3.7/site-packages/habitat_sim/simulator.py", line 122, in __attrs_post_init__
    self.__set_from_config(self.config)
  File "/opt/conda/envs/habitat/lib/python3.7/site-packages/habitat_sim/simulator.py", line 260, in __set_from_config
    self._config_backend(config)
  File "/opt/conda/envs/habitat/lib/python3.7/site-packages/habitat_sim/simulator.py", line 204, in _config_backend
    super().__init__(config.sim_cfg, config.metadata_mediator)
AssertionError: ESP_CHECK failed: Missing (at least) one of scene dataset attributes, stage attributes, or dataset scene attributes for scene 'habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb'.  Likely an invalid scene name.

I'm not sure how to solve this. Could anyone help to get the project running?

ykarmesh commented 2 years ago

Hi @ricardmarsalcastan

Can you confirm that the file habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb is present inside your habitat-challenge folder

hychen-naza commented 2 years ago

Hey @ykarmesh , I want to confirm that the following output of the docker local evaluation is reasonable ./test_locally_objectnav_rgbd.sh --docker-name objectnav_submission

Looks like the agent doesn't move at all, as spl is 0..

[20:24:47:950587]:[Sim] Simulator.cpp(204)::reconfigure : CreateSceneInstance success == true for active scene name : habitat-challenge-data/data/scene_datasets/hm3d/val/00800-TEEsavR23oF/TEEsavR23oF.basis.glb  with renderer.
[20:24:47:950550]:[Sim] Simulator.cpp(464)::instanceStageForSceneAttributes : 
---
The active scene does not contain semantic annotations : activeSemanticSceneID_ = 0  
---
[20:24:47:962659]:[Nav] PathFinder.cpp(386)::build : Building navmesh with 332 x 223 cells
[20:24:48:064274]:[Nav] PathFinder.cpp(656)::build : Created navmesh with 344 vertices 168 polygons
[20:24:48:064312]:[Sim] Simulator.cpp(920)::recomputeNavMesh : reconstruct navmesh successful
2022-04-12 20:24:48,468 distance_to_goal: 8.828277627627054
2022-04-12 20:24:48,468 success: 0.0
2022-04-12 20:24:48,468 spl: 0.0
2022-04-12 20:24:48,468 softspl: 0.011444531018168773
[20:24:48:496740]:[Physics] PhysicsManager.cpp(50)::~PhysicsManager : Deconstructing PhysicsManager
[20:24:48:496789]:[Scene] SceneManager.h(25)::~SceneManager : Deconstructing SceneManager
[20:24:48:496796]:[Scene] SceneGraph.h(25)::~SceneGraph : Deconstructing SceneGraph
[20:24:48:497183]:[Sensor] Sensor.cpp(69)::~Sensor : Deconstructing Sensor
[20:24:48:497452]:[Sensor] Sensor.cpp(69)::~Sensor : Deconstructing Sensor
[20:24:48:497485]:[Scene] SemanticScene.h(47)::~SemanticScene : Deconstructing SemanticScene
[20:24:48:498487]:[Gfx] Renderer.cpp(72)::~Impl : Deconstructing Renderer
[20:24:48:498501]:[Gfx] WindowlessContext.h(17)::~WindowlessContext : Deconstructing WindowlessContext
[20:24:48:500928]:[Sim] Simulator.cpp(65)::~Simulator : Deconstructing Simulator
ykarmesh commented 2 years ago

Hi @hychen-naza ,

This is exactly the output I see on my end as well. The SPL is 0.0 because the agent is just taking random action.

If you want to get better numbers, you should test the ddppo agent, which we detail in point 8 here

ricard-inho commented 2 years ago

Hi @ricardmarsalcastan

Can you confirm that the file habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb is present inside your habitat-challenge folder

@ykarmesh I do have the file in the folder. I reset everything and downloaded all again and still the same error message. This is the log I get before the error message:

realpath: habitat-challenge-data/data/scene_datasets/hm3d: There is no such file or directory.
2022-04-13 08:50:06,941 Initializing dataset ObjectNav-v1
2022-04-13 08:50:07,026 initializing sim Sim-v0
[08:50:07:041174]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : <Dataset>: Proposing JSON name : default.scene_dataset_config.json from original name : default| This file does not exist.
[08:50:07:041290]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DSolid:capsule3DSolid_hemiRings_4_cylRings_1_segments_12_halfLen_0.75_useTexCoords_false_useTangents_false) created and registered.
[08:50:07:041315]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DWireframe:capsule3DWireframe_hemiRings_8_cylRings_1_segments_16_halfLen_1) created and registered.
[08:50:07:041344]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneSolid:coneSolid_segments_12_halfLen_1.25_rings_1_useTexCoords_false_useTangents_false_capEnd_true) created and registered.
[08:50:07:041363]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneWireframe:coneWireframe_segments_32_halfLen_1.25) created and registered.
[08:50:07:041397]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeSolid:cubeSolid) created and registered.
[08:50:07:041415]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeWireframe:cubeWireframe) created and registered.
[08:50:07:041437]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderSolid:cylinderSolid_rings_1_segments_12_halfLen_1_useTexCoords_false_useTangents_false_capEnds_true) created and registered.
[08:50:07:041457]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderWireframe:cylinderWireframe_rings_1_segments_32_halfLen_1) created and registered.
[08:50:07:041484]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereSolid:icosphereSolid_subdivs_1) created and registered.
[08:50:07:041518]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereWireframe:icosphereWireframe_subdivs_1) created and registered.
[08:50:07:041539]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereSolid:uvSphereSolid_rings_8_segments_16_useTexCoords_false_useTangents_false) created and registered.
[08:50:07:041560]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereWireframe:uvSphereWireframe_rings_16_segments_32) created and registered.
[08:50:07:041571]:[Metadata] AssetAttributesManager.cpp(110)::AssetAttributesManager : Built default primitive asset templates : 12
[08:50:07:041844]:[Metadata] SceneDatasetAttributesManager.cpp(35)::createObject : File (default) not found, so new default dataset attributes created  and registered.
[08:50:07:041854]:[Metadata] MetadataMediator.cpp(120)::createSceneDataset : Dataset default successfully created.
[08:50:07:041878]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : <Physics Manager>: Proposing JSON name : ./data/default.physics_config.json from original name : ./data/default.physics_config.json| This file does not exist.
[08:50:07:041896]:[Metadata] PhysicsAttributesManager.cpp(26)::createObject : File (./data/default.physics_config.json) not found, so new default physics manager attributes created and registered.
[08:50:07:041904]:[Metadata] MetadataMediator.cpp(210)::setActiveSceneDatasetName : Attempting to create new dataset ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json
[08:50:07:041915]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : <Dataset>: Proposing JSON name : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json from original name : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json| This file does not exist.
[08:50:07:041938]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DSolid:capsule3DSolid_hemiRings_4_cylRings_1_segments_12_halfLen_0.75_useTexCoords_false_useTangents_false) created and registered.
[08:50:07:041959]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DWireframe:capsule3DWireframe_hemiRings_8_cylRings_1_segments_16_halfLen_1) created and registered.
[08:50:07:041978]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneSolid:coneSolid_segments_12_halfLen_1.25_rings_1_useTexCoords_false_useTangents_false_capEnd_true) created and registered.
[08:50:07:041994]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneWireframe:coneWireframe_segments_32_halfLen_1.25) created and registered.
[08:50:07:042008]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeSolid:cubeSolid) created and registered.
[08:50:07:042021]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeWireframe:cubeWireframe) created and registered.
[08:50:07:042038]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderSolid:cylinderSolid_rings_1_segments_12_halfLen_1_useTexCoords_false_useTangents_false_capEnds_true) created and registered.
[08:50:07:042057]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderWireframe:cylinderWireframe_rings_1_segments_32_halfLen_1) created and registered.
[08:50:07:042073]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereSolid:icosphereSolid_subdivs_1) created and registered.
[08:50:07:042087]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereWireframe:icosphereWireframe_subdivs_1) created and registered.
[08:50:07:042106]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereSolid:uvSphereSolid_rings_8_segments_16_useTexCoords_false_useTangents_false) created and registered.
[08:50:07:042122]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereWireframe:uvSphereWireframe_rings_16_segments_32) created and registered.
[08:50:07:042134]:[Metadata] AssetAttributesManager.cpp(110)::AssetAttributesManager : Built default primitive asset templates : 12
[08:50:07:042340]:[Metadata] SceneDatasetAttributesManager.cpp(35)::createObject : File (./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json) not found, so new default dataset attributes created  and registered.
[08:50:07:042349]:[Metadata] MetadataMediator.cpp(120)::createSceneDataset : Dataset ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json successfully created.
[08:50:07:042356]:[Metadata] MetadataMediator.cpp(217)::setActiveSceneDatasetName : Attempt to create new dataset ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json   succeeded. Currently active dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json
[08:50:07:042363]:[Metadata] MetadataMediator.cpp(175)::setCurrPhysicsAttributesHandle : Old physics manager attributes  changed to ./data/default.physics_config.json successfully.
[08:50:07:042371]:[Metadata] MetadataMediator.cpp(66)::setSimulatorConfiguration : Set new simulator config for scene/stage : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb and dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json which is currently active dataset.
[08:50:07:166344]:[Metadata] MetadataMediator.cpp(297)::getSceneInstanceAttributesByName : Dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json has no preloaded SceneInstanceAttributes or StageAttributes named : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb so loading/creating a new StageAttributes with this name, and then creating a SceneInstanceAttributes with the same name that references this stage.
[08:50:07:166377]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : <Stage Template>: Proposing JSON name : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.stage_config.json from original name : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb| This file does not exist.
[08:50:07:166436]:[Metadata] StageAttributesManager.cpp(86)::registerObjectFinalize : Render asset template handle : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb specified in stage template with handle : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb does not correspond to any existing file or primitive render asset.  Aborting.
[08:50:07:166643]:[Scene] SceneManager.h(25)::~SceneManager : Deconstructing SceneManager
[08:50:07:166940]:[Gfx] Renderer.cpp(72)::~Impl : Deconstructing Renderer
[08:50:07:166950]:[Gfx] WindowlessContext.h(17)::~WindowlessContext : Deconstructing WindowlessContext
ykarmesh commented 2 years ago

The docker is unable to mount the scene dataset as pointed out by this line in your log: realpath: habitat-challenge-data/data/scene_datasets/hm3d: There is no such file or directory.

Can you try running this command from inside the habitat-challenge repository and paste the output here: $(realpath habitat-challenge-data/data/scene_datasets/hm3d)

ricard-inho commented 2 years ago

After running the command I got this message:

realpath: habitat-challenge-data / data / scene_datasets / hm3d: No such files or directories

I attached an image of my folders in the project. The symlink between folders exist right?

Screenshot 2022-04-14 at 08 28 47

ykarmesh commented 2 years ago

I suspect that your habitat-challenge-data/data/scene_datasets/hm3d folder is symlinked to an incorrect path. Can you check if dataDownload/scene_datasets/hm3d folder exists in your machine? It is possible to create symlinks to non-existent files or folders which I believe is the case here (see this issue for more details).