Using HSSD with Habitat Lab

[verityw]

Hello! I'm trying to use HSSD with Habitat Lab, but am running into some issues. I'm running the following code:

config = habitat.get_config("benchmark/nav/objectnav/objectnav_hssd-hab.yaml", overrides=["habitat.dataset.split=val"])
split = "val"
with read_write(config):
    config.habitat.dataset.split = split
    config.habitat.dataset.data_path = f"./habitat-lab/data/datasets/objectnav/hssd-hab/{split}/{split}.json.gz"

env = habitat.Env(config=config)

to create a Habitat lab environment, but am getting the error:

AssertionError: ESP_CHECK failed: No Stage Attributes exists for requested scene 
'data/scene_datasets/hssd-hab/scenes/c/ProcTHOR-Val-682' in currently specified Scene 
Dataset `data/scene_datasets/ai2thor-hab/ai2thor-hab/ai2thor-hab.scene_dataset_config.json`. 
Likely the Scene Dataset Configuration requested was not found and so a new, empty Scene 
Dataset was created. Verify the Scene Dataset Configuration file name used.

From what I understand, the first path in the error is from appending the scene ID (c/ProcTHOR-Val-682) to the config.habitat.dataset.scenes_dir (which is data/scene_datasets/hssd-hab/scenes). This works for HM3D (their scene IDs are like hm3d/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb, which, when prepended with the scenes dir, gets you a path to that basis.glb file). However, that's not the convention that HSSD seems to take.

I thus had the following questions:

• I am getting the config file objectnav_hssd-hab.yaml, but it seems like all the episodes discovered have prefix ProcTHOR (as the scenes are just all the ones from data/datasets/objectnav/hssd-hab/val/content). Is this intentional? Do they correspond to scenes in scene_datasets/hssd-hab or the optional scene_datasets/ai2thor-hab?
• It seems like part of the issue is that, for HM3D, the config.habitat.dataset.scenes_dir joined with the scene ID gets the path to the .basis.glb file defining the scene. However, this does not seem to be the case for ProcTHOR. How do I go about resolving this issue?
• Related to the above, I can't seem to find the equivalent glb file for each ProcTHOR scene. I printed out the deserialized contents of ProcTHOR-Val-0.json.gz and got the following:

  goals_by_category
  ProcTHOR-Val-0_Bed [List of info of all beds in scene]
  ProcTHOR-Val-0_Chair [List of info of all chairs in scene]
  ...
  episodes [List of all episodes in scene]
  category_to_task_category_id
  Chair 0
  Bed 1
  ...
  category_to_scene_annotation_category_id
  Chair 0
  Bed 1
  ...

  Printing out each of the episodes, I get something like:

  episode_id 0
  scene_id c/ProcTHOR-Val-0
  scene_dataset_config data/scene_datasets/ai2thor-hab/ai2thor-hab/ai2thor-hab.scene_dataset_config.json
  additional_obj_config_paths []
  start_position [-13.57502, 0.14062, 7.28813]
  start_rotation [0, 0.51301, 0, 0.85839]
  info
  geodesic_distance 1.05324, euclidean_distance 1.00184, closest_goal_object_id 98
  goals []
  start_room None
  shortest_paths None
  object_category Bed

  However, none of this seems to specify which of the glb files to use. I guess scene_datasets/ai2thor-hab/ai2thor-hab/configs/stages/ProcTHOR/c/ProcTHOR-Val-0.stage_config.json does start with "../../../../assets/stages/ProcTHOR/c/ProcTHOR-Val-0.glb", but I'm not sure that's right + dunno how that can be found and used by Habitat Lab.

Any help with the above questions or general method for using Habitat Lab with HSSD would be extremely helpful. Please let me know if any additional information is required, and I'd be glad to provide it!

[mukulkhanna]

Hello @verityw,

First and foremost, it seems like you are loading the HSSD task config, but trying to load ProcTHOR scenes. ProcTHOR scenes have a separate task config (here). Please use this one.

Also, make sure the scene dataset config file (e.g. data/scene_datasets/ai2thor-hab/ai2thor-hab/ai2thor-hab.scene_dataset_config.json) for the scene dataset exists for the scenes to be loaded correctly.

[verityw]

Thanks for the reply!

How can I load the HSSD files then? It seems like the download links here for HSSD and ProcTHOR are identical, as it was changed 3 weeks ago. I initially didn't notice this and instead unzipped the download for ProcTHOR to data/datasets/objectnav/hssd-hab, so when I used the config file for HSSD, it locked in that path and found the ProcTHOR dataset info.

Edit: the download link has been fixed!

I have confirmed that the ai2thor-hab.scene_dataset_config.json exists. I see now that the c/Procthor-whatever is the prefix for the scene config files in habitat-lab/data/scene_datasets/ai2thor-hab/ai2thor-hab/configs/scenes/ProcTHOR. Additionally, while I am getting a similar error as before, I think it may just be a pathing issue (since all the paths start with data but I am working in the same directory as habitat-lab, so ./habitat-lab/ may need to be prepended to everything). I will message again tomorrow when I've tried this.