search

facebookresearch / habitat-challenge

Code for the habitat challenge
https://aihabitat.org
MIT License
303 stars 57 forks source link

Semantic annotations not getting loaded #103

Closed nnsriram97 closed 2 years ago

nnsriram97 commented 2 years ago

Hi, I am trying to visualize an ObjectNav agent but there seems to be an error in TopDownMap measure when it is trying to draw goals:

File ".../habitat-challenge-dev/habitat/tasks/nav/nav.py", line 784, in _draw_goals_aabb
    sem_scene.objects[object_id].id.split("_")[-1]
IndexError: list index out of range

Seems like semantic annotations are not getting loaded but I have the complete dataset.

[22:35:57:534470]:[Sim] Simulator.cpp(432)::instanceStageForSceneAttributes : Successfully loaded stage named : habitat-challenge-data/data/scene_datasets/hm3d/val/00839-zt1RVoi7PcG/zt1RVoi7PcG.basis.glb
[22:35:57:534517]:[Sim] Simulator.cpp(464)::instanceStageForSceneAttributes :
---
The active scene does not contain semantic annotations : activeSemanticSceneID_ = 0
---

Am I missing something here?

ykarmesh commented 2 years ago

Hi @nnsriram97, are you trying to do this from inside the docker? Confirming this because we don't support loading of semantic annotations from inside the docker container.

nnsriram97 commented 2 years ago

Nope, I am not running things on the docker.

Ending2015a commented 2 years ago

I also have this issue, but I;m running inside the docker.

The datasets are downloaded via command line python -m habitat_sim .... from inside the docker. So I guess the paths are properly set. However when I access the env.sim.semantic_annotations() after building the simulator, it shows nothing.

Here is the log:

logs 
2022-03-20 10:06:38,034 Initializing dataset ObjectNav-v1
2022-03-20 10:06:38,135 initializing sim Sim-v0
[10:06:38:151523]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : : Proposing JSON name : default.scene_dataset_config.json from original name : default| This file does not exist.
[10:06:38:151623]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DSolid:capsule3DSolid_hemiRings_4_cylRings_1_segments_12_halfLen_0.75_useTexCoords_false_useTangents_false) created and registered.
[10:06:38:151672]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DWireframe:capsule3DWireframe_hemiRings_8_cylRings_1_segments_16_halfLen_1) created and registered.
[10:06:38:151711]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneSolid:coneSolid_segments_12_halfLen_1.25_rings_1_useTexCoords_false_useTangents_false_capEnd_true) created and registered.
[10:06:38:151745]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneWireframe:coneWireframe_segments_32_halfLen_1.25) created and registered.
[10:06:38:151781]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeSolid:cubeSolid) created and registered.
[10:06:38:151809]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeWireframe:cubeWireframe) created and registered.
[10:06:38:151840]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderSolid:cylinderSolid_rings_1_segments_12_halfLen_1_useTexCoords_false_useTangents_false_capEnds_true) created and registered.
[10:06:38:151872]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderWireframe:cylinderWireframe_rings_1_segments_32_halfLen_1) created and registered.
[10:06:38:151907]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereSolid:icosphereSolid_subdivs_1) created and registered.
[10:06:38:151938]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereWireframe:icosphereWireframe_subdivs_1) created and registered.
[10:06:38:151973]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereSolid:uvSphereSolid_rings_8_segments_16_useTexCoords_false_useTangents_false) created and registered.
[10:06:38:152011]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereWireframe:uvSphereWireframe_rings_16_segments_32) created and registered.
[10:06:38:152042]:[Metadata] AssetAttributesManager.cpp(110)::AssetAttributesManager : Built default primitive asset templates : 12
[10:06:38:152335]:[Metadata] SceneDatasetAttributesManager.cpp(35)::createObject : File (default) not found, so new default dataset attributes created  and registered.
[10:06:38:152353]:[Metadata] MetadataMediator.cpp(120)::createSceneDataset : Dataset default successfully created.
[10:06:38:152368]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : : Proposing JSON name : ./data/default.physics_config.json from original name : ./data/default.physics_config.json| This file does not exist.
[10:06:38:152386]:[Metadata] PhysicsAttributesManager.cpp(26)::createObject : File (./data/default.physics_config.json) not found, so new default physics manager attributes created and registered.
[10:06:38:152400]:[Metadata] MetadataMediator.cpp(210)::setActiveSceneDatasetName : Attempting to create new dataset ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json
[10:06:38:152413]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : : Proposing JSON name : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json from original name : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json| This file does not exist.
[10:06:38:152449]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DSolid:capsule3DSolid_hemiRings_4_cylRings_1_segments_12_halfLen_0.75_useTexCoords_false_useTangents_false) created and registered.
[10:06:38:152484]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (capsule3DWireframe:capsule3DWireframe_hemiRings_8_cylRings_1_segments_16_halfLen_1) created and registered.
[10:06:38:152511]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneSolid:coneSolid_segments_12_halfLen_1.25_rings_1_useTexCoords_false_useTangents_false_capEnd_true) created and registered.
[10:06:38:152545]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (coneWireframe:coneWireframe_segments_32_halfLen_1.25) created and registered.
[10:06:38:152573]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeSolid:cubeSolid) created and registered.
[10:06:38:152599]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cubeWireframe:cubeWireframe) created and registered.
[10:06:38:152634]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderSolid:cylinderSolid_rings_1_segments_12_halfLen_1_useTexCoords_false_useTangents_false_capEnds_true) created and registered.
[10:06:38:152676]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (cylinderWireframe:cylinderWireframe_rings_1_segments_32_halfLen_1) created and registered.
[10:06:38:152715]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereSolid:icosphereSolid_subdivs_1) created and registered.
[10:06:38:152750]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (icosphereWireframe:icosphereWireframe_subdivs_1) created and registered.
[10:06:38:152785]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereSolid:uvSphereSolid_rings_8_segments_16_useTexCoords_false_useTangents_false) created and registered.
[10:06:38:152820]:[Metadata] AssetAttributesManager.cpp(121)::createObject : Asset attributes (uvSphereWireframe:uvSphereWireframe_rings_16_segments_32) created and registered.
[10:06:38:152850]:[Metadata] AssetAttributesManager.cpp(110)::AssetAttributesManager : Built default primitive asset templates : 12
[10:06:38:153108]:[Metadata] SceneDatasetAttributesManager.cpp(35)::createObject : File (./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json) not found, so new default dataset attributes created  and registered.
[10:06:38:153126]:[Metadata] MetadataMediator.cpp(120)::createSceneDataset : Dataset ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json successfully created.
[10:06:38:153137]:[Metadata] MetadataMediator.cpp(217)::setActiveSceneDatasetName : Attempt to create new dataset ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json   succeeded. Currently active dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json
[10:06:38:153146]:[Metadata] MetadataMediator.cpp(175)::setCurrPhysicsAttributesHandle : Old physics manager attributes  changed to ./data/default.physics_config.json successfully.
[10:06:38:153160]:[Metadata] MetadataMediator.cpp(66)::setSimulatorConfiguration : Set new simulator config for scene/stage : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb and dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json which is currently active data
set.
[10:06:38:221467]:[Metadata] MetadataMediator.cpp(297)::getSceneInstanceAttributesByName : Dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json has no preloaded SceneInstanceAttributes or StageAttributes named : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb s
o loading/creating a new StageAttributes with this name, and then creating a SceneInstanceAttributes with the same name that references this stage.
[10:06:38:221505]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : : Proposing JSON name : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.stage_config.json from original name : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4
TFT35.basis.glb| This file does not exist.
[10:06:38:221586]:[Metadata] AbstractObjectAttributesManagerBase.h(211)::createObject : File (habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb) exists but is not a recognized config filename extension, so new default Stage Template attributes created and registered.
[10:06:38:221632]:[Metadata] SceneDatasetAttributes.cpp(47)::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Stage Attributes 'habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb' specified in Scene Attributes exists in dataset library.
[10:06:38:221678]:[Metadata] SceneDatasetAttributes.cpp(84)::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Lighting Layout Attributes 'no_lights' specified in Scene Attributes but does not exist in dataset, so creating.
[10:06:38:221696]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : : Proposing JSON name : no_lights.lighting_config.json from original name : no_lights| This file does not exist.
[10:06:38:221716]:[Metadata] LightLayoutAttributesManager.cpp(36)::createObject : File (no_lights) not found, so new default light layout attributes created and registered.
[10:06:38:221733]:[Sim] Simulator.cpp(236)::createSceneInstance : Navmesh file location in scene instance : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.navmesh
[10:06:38:221754]:[Sim] Simulator.cpp(241)::createSceneInstance : Loading navmesh from habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.navmesh
[10:06:38:221925]:[Sim] Simulator.cpp(243)::createSceneInstance : Navmesh Loaded.
[10:06:38:221953]:[Scene] SceneGraph.h(85)::createDrawableGroup : Created DrawableGroup:
[10:06:38:221970]:[Assets] ResourceManager.cpp(298)::loadSemanticSceneDescriptor : SceneInstance : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb proposed Semantic Scene Descriptor filename : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.scn
[10:06:38:221990]:[Assets] ResourceManager.cpp(345)::loadSemanticSceneDescriptor : SSD File Naming Issue! Neither SceneInstanceAttributes-provided name : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.scn  nor constructed filename : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcoj
b4TFT35/info_semantic.json exist on disk.
[10:06:38:222052]:[Assets] ResourceManager.cpp(1647)::ConfigureImporterManagerGLExtensions : Importing Basis files as BC7.
[10:06:38:222288]:[Sim] Simulator.cpp(301)::createSceneInstance : Using scene instance-specified Light key : - no_lights -
[10:06:38:222310]:[Metadata] MetadataMediator.cpp(66)::setSimulatorConfiguration : Set new simulator config for scene/stage : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb and dataset : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json which is currently active data
set.
[10:06:38:222336]:[Sim] Simulator.cpp(405)::instanceStageForSceneAttributes : Start to load stage named : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb with render asset : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb and collision asset : habi
tat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb
[10:06:38:222379]:[Assets] ResourceManager.cpp(689)::createStageAssetInfosFromAttributes : Frame :{"up":[0,0,1],"front":[0,1,0],"origin":[0,0,0]} for render mesh named : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb|No Semantic asset info specified.
[10:06:38:222392]:[Assets] ResourceManager.cpp(473)::loadStage : Not loading semantic mesh
[10:06:38:222404]:[Assets] ResourceManager.cpp(507)::loadStage : Start load render asset habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb .
[10:06:38:222413]:[Assets] ResourceManager.cpp(970)::loadStageInternal : Attempting to load stage habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb
[10:06:38:222432]:[Assets] ResourceManager.cpp(853)::loadRenderAsset : Loading general asset named: habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb
[10:06:38:222448]:[Assets] ResourceManager.cpp(1647)::ConfigureImporterManagerGLExtensions : Importing Basis files as BC7.
[10:06:40:477064]:[Assets] ResourceManager.cpp(2081)::loadMaterials : Building 65 materials for asset named 'wcojb4TFT35.basis.glb' :
[10:06:40:477137]:[Assets] ResourceManager.cpp(2186)::loadMaterials : Idx 00:Flat.
[10:06:40:477153]:[Assets] ResourceManager.cpp(2186)::loadMaterials : Idx 01:Flat.
[10:06:40:477165]:[Assets] ResourceManager.cpp(2186)::loadMaterials : Idx 02:Flat.
.....
[10:06:40:478276]:[Assets] ResourceManager.cpp(2186)::loadMaterials : Idx 64:Flat.
[10:06:40:799698]:[Sim] Simulator.cpp(432)::instanceStageForSceneAttributes : Successfully loaded stage named : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb
[10:06:40:799741]:[Sim] Simulator.cpp(464)::instanceStageForSceneAttributes :
---
The active scene does not contain semantic annotations : activeSemanticSceneID_ = 0
---
[10:06:40:799838]:[Sim] Simulator.cpp(204)::reconfigure : CreateSceneInstance success == true for active scene name : habitat-challenge-data/data/scene_datasets/hm3d/val/00802-wcojb4TFT35/wcojb4TFT35.basis.glb  with renderer.
[10:06:40:827126]:[Nav] PathFinder.cpp(386)::build : Building navmesh with 215 x 415 cells
[10:06:40:998175]:[Nav] PathFinder.cpp(656)::build : Created navmesh with 571 vertices 289 polygons
[10:06:40:998238]:[Sim] Simulator.cpp(920)::recomputeNavMesh : reconstruct navmesh successful
2022-03-20 10:06:41,006 Initializing task ObjectNav-v1

Hi @nnsriram97, are you trying to do this from inside the docker? Confirming this because we don't support loading of semantic annotations from inside the docker container.

May I ask the reason why we cant load semantic annotations inside the docker? Is it related to habitat sim version? Currently I'm using the self-built docker image with the latest version of habitat sim and habitat lab.

dhruvbatra commented 2 years ago

May I ask the reason why we cant load semantic annotations inside the docker?

Because these are challenge submission / testing dockers. Your agent needs to operate solely based on RGBD at test-time.

Ending2015a commented 2 years ago

Because these are challenge submission / testing dockers. Your agent needs to operate solely based on RGBD at test-time.

@dhruvbatra you mean loading annotations in "testing_2022_habitat_base_docker" docker is not allowed. Now I'm using the docker built with this Dockerfile from habitat_lab. The habitat sim and habitat lab are cloned from the latest main branch. But I still cannot load the semantic annotations (the logs are provided as above). Cloud you tell me how to load annotations in the custom build docker? Thank you.

ykarmesh commented 2 years ago

@nnsriram97 Semantic annotations are not loaded by default for ObjectNav. To get the semantic annotations loaded, you need to add SEMANTIC_SENSOR in the list of sensors in the baseline config. For eg, if you are using the ddppo_objectnav.yaml file, you should change line 13 to look like this: 

SENSORS: ["DEPTH_SENSOR", "RGB_SENSOR", "SEMANTIC_SENSOR"]

Semantic annotations should start loading after that.

nnsriram97 commented 2 years ago

Hi @ykarmesh, I still face the same error.

To give some more context, I had installed habitat-sim with conda install habitat-sim withbullet headless -c conda-forge -c aihabitat as one on the challenge repository (conda install habitat-sim-challenge-2022 -c conda-forge -c aihabitat) did not work for me. And I used the challenge-2022 branch for habitat-lab. Thanks!

srama2512 commented 2 years ago

@nnsriram97 - I was able to successfully evaluate the ddppo baseline checkpoint on HM3D minival with video saving. It is possible that the issue may be with the conda installation. We're looking into it. In the meanwhile, here are the steps I took for evaluation (NOTE: I'm not loading semantic annotations here):

python habitat_baselines/run.py \ --exp-config habitat_baselines/config/objectnav/ddppo_objectnav_hm3d.yaml \ --run-type eval \ EVAL.SPLIT "val_mini" \ EVAL_CKPT_PATH_DIR $EXPT_DIR/ddppo_objectnav_habitat2022_challenge_baseline_v1.pth \ NUM_ENVIRONMENTS 2 \ TENSORBOARD_DIR $EXPT_DIR/eval_minival \ LOG_FILE $EXPT_DIR/log_eval_minival.txt \ VIDEO_OPTION "['disk']" \ VIDEO_DIR $EXPT_DIR/video_minival

nnsriram97 commented 2 years ago

Thanks, @srama2512. I only had an issue with _draw_goals_aabb (here) in TopDownMap while saving the video as it was accessing semantic annotations. Putting TOP_DOWN_MAP.DRAW_GOAL_AABBS False worked for me.

Ending2015a commented 2 years ago

@dhruvbatra @srama2512 I found the solution. The following log printed by habitat_sim

Proposing JSON name : ./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json from original name :
./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json| This file does not exist.

shows that habitat_sim searches configuration files in ./data, so I create a symbolic link inside the docker

ln -s /habitat-challenge-data/data /data

Then everything works. Thank you.

Screenshot from 2022-03-24 11-39-01

alstar8 commented 2 years ago

@Ending2015a Could you please provide the script with configs to reproduce it, I do as you write, but simulator still can not find the semantic files

Ending2015a commented 2 years ago

@Ending2015a Could you please provide the script with configs to reproduce it, I do as you write, but simulator still can not find the semantic files

Hi @alstar8 , does your simulator load the scene correctly? If so, I think maybe you are missing some config files or maybe your dataset organization is not correct. If you are following the README to download datasets (you have to set the --data-path to habitat-challenge-data/data/):

python -m habitat_sim.utils.datasets_download --username <api-token-id> --password <api-token-secret> --uids hm3d_val --data-path /habitat-challenge-data/data/

The dataset organization may look like this:

habitat-challenge-data/data/
├── scene_datasets
│   └── hm3d -> /habitat-challenge-data/data/versioned_data/hm3d-1.0/hm3d
└── versioned_data
    └── hm3d-1.0
        ├── hm3d
        │   ├── hm3d_annotated_basis.scene_dataset_config.json
        │   ├── minival
        │   ├── train
        │   └── val
        ├── minival-configs-files.json.gz
        ├── minival-habitat-files.json.gz
        ├── minival-semantic-annots-files.json.gz
        ├── minival-semantic-configs-files.json.gz
        ├── train-configs-files.json.gz
        ├── train-habitat-files.json.gz
        ├── train-semantic-annots-files.json.gz
        ├── train-semantic-configs-files.json.gz
        ├── val-configs-files.json.gz
        ├── val-habitat-files.json.gz
        ├── val-semantic-annots-files.json.gz
        └── val-semantic-configs-files.json.gz

And the episode data:

habitat-challenge-data/
├── data
│   ├── scene_datasets
│   └── versioned_data
├── objectgoal_hm3d
│   ├── train
│   ├── val
│   └── val_mini
└── objectnav_hm3d_v1.zip

As @ykarmesh mentioned, make sure you add the SEMANTIC_SENSOR to your config file. Then you are ready for loading the semantic data! Just creating the link ln - s habitat-challenge-data/data/ data/ or create a volumn docker run ... -v $(pwd)/habitat-challenge-data/data:/data ... for the docker. Then the semantic data will be loaded automatically. Also mention that I'm using the latest main branch of habitat lab and habitat sim to build the docker for training agents. I'm not sure if the challenge docker or the challenge-2022 branch for habitat_lab works.

alstar8 commented 2 years ago

@Ending2015a @dhruvbatra @ykarmesh Thank you for the response. I've got the same data structure and have a link to /data. Also I'am using docker with the latest main branch of habitats. I load the config file as

config_env = habitat.get_config("challenge_objectnav2022.local.rgbd.yaml")
config_env.defrost()
config_env.SIMULATOR.AGENT_0.SENSORS.append("SEMANTIC_SENSOR")
config_env.DATASET.SCENES_DIR = '/data/scene_datasets/'
config_env.DATASET.DATA_PATH = '/data/datasets/objectnav/hm3d/{split}/{split}.json.gz'
config_env.DATASET.SPLIT = 'val'
config_env.freeze()

If I added to the config line

config_env.SIMULATOR.SCENE_DATASET = "/data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json"

The env loading starts to take forever

At the end I've got an error

2022-03-25 11:11:07,191 Initializing dataset ObjectNav-v1
2022-03-25 11:11:08,355 initializing sim Sim-v0
[11:11:08:473343]:[Assets] ResourceManager.cpp(345)::loadSemanticSceneDescriptor : SSD File Naming Issue! Neither SceneInstanceAttributes-provided name : /data/scene_datasets/hm3d/val/00878-XB4GS9ShBRE/XB4GS9ShBRE.basis.scn  nor constructed filename : /data/scene_datasets/hm3d/val/00878-XB4GS9ShBRE/info_semantic.json2022-03-25 11:11:09,491 Initializing task ObjectNav-v1
[11:11:08:368300]:[Metadata] AttributesManagerBase.h(352)::createFromJsonOrDefaultInternal : <Dataset>: Proposing JSON name : default.scene_dataset_config.json from original name : default| This file does not exist.
[11:11:08:368418]:[Metadata] 
[11:11:09:396806]:[Sim] Simulator.cpp(464)::instanceStageForSceneAttributes : 
---
The active scene does not contain semantic annotations : activeSemanticSceneID_ = 0  
---

And I've got no idea what the info_semantic.json or XB4GS9ShBRE.basis.scn files are

KirillMouraviev commented 2 years ago

The solution for me was to move file hm3d_annotated_basis.scene_dataset_config.json into directory <path_to_data>/scene_datasets/hm3d, and to rebuild habitat-sim with latest version of branch main. I executed Habitat with such setup:

config_env = get_config(
    "ddppo_objectnav.yaml", ["BASE_TASK_CONFIG_PATH", 'challenge_objectnav2022.local.rgbd.yaml']).clone()
config_env.defrost()
config_env.TASK_CONFIG.SIMULATOR.AGENT_0.SENSORS.append("SEMANTIC_SENSOR")

habitat_path = '/home/kirill/habitat-lab/data'
config_env.TASK_CONFIG.DATASET.SCENES_DIR = os.path.join(habitat_path, 'scene_datasets/')
config_env.TASK_CONFIG.DATASET.DATA_PATH = os.path.join(habitat_path,
                                                       'datasets/objectnav_hm3d_v1/val/val.json.gz')
config_env.TASK_CONFIG.SIMULATOR.SCENE_DATASET = os.path.join(habitat_path,
    "scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json")

config_env.TASK_CONFIG.TASK.MEASUREMENTS.append("TOP_DOWN_MAP")
config_env.TASK_CONFIG.TASK.SENSORS.append("HEADING_SENSOR")
config_env.TASK_CONFIG.DATASET.CONTENT_SCENES = ['mv2HUxq3B53']

config_env.freeze()
xuai05 commented 2 years ago

When I run python -m habitat_sim.utils.datasets_download --username --password --uids hm3d_val --data-path /habitat-challenge-data/data/ , the following error pops up.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    82  100    82    0     0     10      0  0:00:08  0:00:08 --:--:--    18
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0
100 10533  100 10533    0     0    981      0  0:00:10  0:00:10 --:--:--     0
Traceback (most recent call last):
  File "/data/huxiaobo/anaconda3/envs/habitat/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/data/huxiaobo/anaconda3/envs/habitat/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/data/huxiaobo/anaconda3/envs/habitat/lib/python3.7/site-packages/habitat_sim/utils/datasets_download.py", line 688, in <module>
    main(sys.argv[1:])
  File "/data/huxiaobo/anaconda3/envs/habitat/lib/python3.7/site-packages/habitat_sim/utils/datasets_download.py", line 683, in main
    uid, data_path, args.username, args.password, replace
  File "/data/huxiaobo/anaconda3/envs/habitat/lib/python3.7/site-packages/habitat_sim/utils/datasets_download.py", line 520, in download_and_place
    with tarfile.open(data_path + package_name, "r:*") as tar_ref:
  File "/data/huxiaobo/anaconda3/envs/habitat/lib/python3.7/tarfile.py", line 1580, in open
    raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

But when I go to get the example data, the acquisition program runs without problems. Can someone tell me how to resolve it?

arjung128 commented 2 years ago

@xuai05 I'm getting this when I install habitat-sim with conda install habitat-sim-challenge-2022 -c conda-forge -c aihabitat. It seems to go away when I install habitat-sim with conda install habitat-sim withbullet headless -c conda-forge -c aihabitat. But with this workaround, the semantic annotations don't seem to get downloaded...

ybgdgh commented 2 years ago

I meet problems with downloading the semantic-annots files. The download of hm3d_val_habitat and hm3d_val_configs work well but when the script runs the third part about hm3d-val-semantic-annots-v0.1, it is always failed like the following errors:


** Resuming transfer from byte position 320
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1347  100  1347    0     0   7610      0 --:--:-- --:--:-- --:--:--  7610
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
Traceback (most recent call last):
  File "/home/ybg/anaconda3/envs/habitat21data/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ybg/anaconda3/envs/habitat21data/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ybg/anaconda3/envs/habitat21data/lib/python3.7/site-packages/habitat_sim-0.2.1-py3.7-linux-x86_64.egg/habitat_sim/utils/datasets_download.py", line 688, in <module>
    main(sys.argv[1:])
  File "/home/ybg/anaconda3/envs/habitat21data/lib/python3.7/site-packages/habitat_sim-0.2.1-py3.7-linux-x86_64.egg/habitat_sim/utils/datasets_download.py", line 683, in main
    uid, data_path, args.username, args.password, replace
  File "/home/ybg/anaconda3/envs/habitat21data/lib/python3.7/site-packages/habitat_sim-0.2.1-py3.7-linux-x86_64.egg/habitat_sim/utils/datasets_download.py", line 506, in download_and_place
    subprocess.check_call(shlex.split(download_command))
  File "/home/ybg/anaconda3/envs/habitat21data/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['curl', '--continue-at', '-', '--location', '--user', 'f7ed2bcd9b643a4b:b837aec424eb1ae3d4bde0e3b3e1d0d4', 'https://api.matterport.com/resources/habitat/hm3d-val-semantic-annots-v0.1.tar.gz', '-o', '/home/ybg/Project/challenge/habitat-challenge2022/habitat-challenge-data/data/hm3d-val-semantic-annots-v0.1.tar.gz']' returned non-zero exit status 33.

When i just run 

curl --continue-at - --location --user f7ed2bcd9b643a4b:b837aec424eb1ae3d4bde0e3b3e1d0d4 https://api.matterport.com/resources/habitat/hm3d-val-semantic-annots-v0.1.tar.gz -o /home/ybg/Project/challenge/habitat-challenge2022/habitat-challenge-data/data/hm3d-val-semantic-annots-v0.1.tar.gz

it still shows:

** Resuming transfer from byte position 320
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1355  100  1355    0     0   6215      0 --:--:-- --:--:-- --:--:--  6215
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.

Anyone knows how can i solve this problem?

ykarmesh commented 2 years ago

We recently sent out this msg to all the participants:

Due to unforeseen circumstances surrounding the availability of the HM3D-Semantic v0.1 dataset we have decided to extend the deadline for the Habitat-Challenge 2022, tentatively by three months. We realize many participants are impacted by this and apologize for the disruption. We are working on making the data available again and will follow up with a more exact updated challenge timeline soon. In the meantime, the Test Standard phase leaderboard continues to be open, and you are welcome to report results from it in your papers. To not disadvantage anyone who has already submitted, we will reset the Test Challenge leaderboard submission count. We plan to provide an update about the challenge at the CVPR 2022 Embodied AI workshop, and to announce challenge results at the upcoming NeurIPS 2022 conference.

Apologies for the inconvenience.

ykarmesh commented 2 years ago

The HM3D-Semantic data is available for downloading again.

taokong commented 2 years ago

We recently sent out this msg to all the participants:

Due to unforeseen circumstances surrounding the availability of the HM3D-Semantic v0.1 dataset we have decided to extend the deadline for the Habitat-Challenge 2022, tentatively by three months. We realize many participants are impacted by this and apologize for the disruption. We are working on making the data available again and will follow up with a more exact updated challenge timeline soon. In the meantime, the Test Standard phase leaderboard continues to be open, and you are welcome to report results from it in your papers. To not disadvantage anyone who has already submitted, we will reset the Test Challenge leaderboard submission count. We plan to provide an update about the challenge at the CVPR 2022 Embodied AI workshop, and to announce challenge results at the upcoming NeurIPS 2022 conference.

Apologies for the inconvenience.

HI @ykarmesh, NeurIPS 2022 will begin three months after the submission DDL. Will you plan to release the ObjectNav challenge results earlier since it has been closed?

ykarmesh commented 2 years ago

Hi @taokong, great question! We are planning to release the results by the 1st week of October. You should be able to it on our twitter handle @ai_habitat and website