Replica PointNav dataset

[mbanani]

I am currently setting the Replica dataset as instructed here, but I was unable to find the pointnav dataset for replica. Habitat-api seems to offer PointNav datasets for Gibson and Matterport3D, but not Replica. I wasn't sure if I missed the link somewhere, or if I should be generating the dataset independently. Thank you for your help!

[oawiles]

Yeah you download Replica independently (it's linked under the Replica heading)

[mbanani]

@oawiles, thank you for the very quick response.

Sorry if my first question was vague; I did download the replica dataset, but this line refers to a pointnav dataset for the 3 splits. Habitat-api provides pointnav datasets for Matterport3D but they don't provide one for Replica.

I can use habitat-api to generate a pointnav dataset for Replica, but it seems to involve some degree of random sampling, and I wanted to make sure I was using the same dataset as the paper for direct comparison.

I would appreciate any clarification if I am missing something. Thanks again for your help!

[oawiles]

Oh I see! Sorry I was confused. Yes you should generate your own but actually you may not need to: The pointnav stuff isn't really used beyond it being how we record the scenes that exist in the dataset. But then we saved this to one scene per environment here. And I have these in the github repo here. So you should be able to just use those saved zip files and the point nav datasets shouldn't matter.

(sorry this is confusing -- I don't think Habitat was really supposed to be used for view synthesis so I kind of hacked around their assumption it's for controlling a robot in an environment).

[mbanani]

Thanks a lot for the clarification; that makes a lot of sense. I noticed it wasn't really being used in the code, but wasn't sure if habitat-sim was somehow using it under the hood. I think I still need to generate it because habitat-sim expects it, if I find a way around that, I will update this issue.

Update: After looking more into using the one_scene_per_environment datasets, I realized that you were making my life much easier. I could simply comment out L131-148 here, and have the data_dir point to the appropriate directory.

One issue that I ran into was that the datasets you generated had some hardcoded paths; this was easily fixable by updating the paths as done in L47 and L157.

Thanks again!

[oskar0812]

@mbanani @oawiles Hi,i have some same problems like this. When I run the code to generate the Replica dataset, the program reports an error:

Loading dataset replica ...
Loading train dataset ....
<data.habitat_data.HabitatImageGenerator object at 0x7f8e3c566e50>
Restarting image_generator.... with seed 0 in train mode? True
gpu_id 0
Traceback (most recent call last):
  File "/train.py", line 23, in <module>
    images = train_set['images']
  File "/data/habitat_data.py", line 119, in __getitem__
    self.__restart__()
  File "/data/habitat_data.py", line 35, in __restart__
    self.image_generator = RandomImageGenerator(  # 传入参数
  File "/data/create_rgb_dataset.py", line 113, in __init__
    config = make_config(
  File "/data/create_rgb_dataset.py", line 63, in make_config
    config = get_config(config)
  File "/habitat-lab/habitat-lab/habitat/config/default.py", line 126, in get_config
    config_path = get_full_config_path(config_path, configs_dir)
  File "/habitat-lab/habitat-lab/habitat/config/default.py", line 44, in get_full_config_path
    raise RuntimeError(f"No file found for config '{config_path}'")
RuntimeError: No file found for config '/private/home/ow045820/projects/habitat/habitat-api/configs/tasks/pointnav_rgbd.yaml'

I don't know how to actually not use pointnav stuff.But I seem to have to pass a pointnav file in. Can you give me some advice? thank you very much

Oh I see! Sorry I was confused. Yes you should generate your own but actually you may not need to: The pointnav stuff isn't really used beyond it being how we record the scenes that exist in the dataset. But then we saved this to one scene per environment here. And I have these in the github repo here. So you should be able to just use those saved zip files and the point nav datasets shouldn't matter.

(sorry this is confusing -- I don't think Habitat was really supposed to be used for view synthesis so I kind of hacked around their assumption it's for controlling a robot in an environment).

[mbanani]

One thing I noticed is that you're trying to load a config from the original path (on Olivia's machine) which is set here. You need to make sure to update the paths to your own machine and where you download the dataset. I don't recall what I specifically did to circumvent the Replica issue beyond what I wrote above as it's been three years. Sorry I can't help more. Good luck!

[oskar0812]

One thing I noticed is that you're trying to load a config from the original path (on Olivia's machine) which is set here. You need to make sure to update the paths to your own machine and where you download the dataset. I don't recall what I specifically did to circumvent the Replica issue beyond what I wrote above as it's been three years. Sorry I can't help more. Good luck!

Thank you for your reply! I understand what you said that I need to update the paths. But like what you wrote above, the replica dataset I use I don't know where to find its pointnav. And like Olivia @oawiles said, it seems that pointnav is not actually used. Is there any way to avoid calling this file？

[oskar0812]

I realize that pointnav is just a base config file and has nothing to do with Replica. But when I choose the correct path, the program will report a new error.

Loading dataset replica ...
Loading train dataset ....
<data.habitat_data.HabitatImageGenerator object at 0x7f343e352e20>
Restarting image_generator.... with seed 0 in train mode? True
gpu_id 0
Traceback (most recent call last):
  File "train.py", line 23, in <m
    images = train_set['images']
  File "/data/habitat_data.py", li in __getitem__
    self.__restart__()
  File "/data/habitat_data.py", liin __restart__
    self.image_generator = RandomImageGenerator(  
  File "/data/create_rgb_dataset.pe 113, in __init__
    config = make_config(
  File "/data/create_rgb_dataset.pe 63, in make_config
    config = get_config(config)
  File "/habitat-lab/habitat-lab/habitat/config/default.py", li in get_config
    return patch_config(cfg)
  File "/habitat-lab/habitat-lab/habitat/config/default.py", liin patch_config
    sim_config = cfg.habitat.simulator
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/dictconfig.py", line 359, in __getattr__
    self._format_and_raise(key=key, value=None, cause=e)
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/base.py", line 231, in _format_and_raise
    format_and_raise(
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/_utils.py", line 819, in format_and_raise
    _raise(ex, cause)
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/_utils.py", line 797, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/dictconfig.py", line 351, in __getattr__
    return self._get_impl(
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/dictconfig.py", line 442, in _get_impl
    node = self._get_child(
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/basecontainer.py", line 73, in _get_child
    child = self._get_node(
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/dictconfig.py", line 475, in _get_node
    self._validate_get(key)
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/dictconfig.py", line 164, in _validate_get
    self._format_and_raise(
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/base.py", line 231, in _format_and_raise
    format_and_raise(
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/_utils.py", line 899, in format_and_raise
    _raise(ex, cause)
  File "/anaconda3/envs/habitat-lab/lib/python3.9/site-packages/of/_utils.py", line 797, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
omegaconf.errors.ConfigAttributeError: Key 'habitat' is not in struct
    full_key: habitat
    object_type=dict