Issue with Point Navigation Success Rate Using Pre-trained Pointnav Model

[Zeying-Gong]

Dear VLFM Team,

I have been working on replicating the Habitat simulation part of the vlfm code. While the success rate for object goal navigation matches the results reported in the paper, I have felt confused with the point navigation task.

Specifically, using the pre-trained model, I only achieved an average success rate of approximately 46% over three runs. This value is significantly lower than expected, as I anticipated a success rate of over 90% for point navigation. However, this lower success rate seems not to affect the success rate for the final object goal navigation.

As the paper does not report the success rate for point navigation with the pre-trained model, I wanted to confirm if this success rate is close to the truth. Could you provide any clarifications regarding to the expected performance of the pre-trained model?

Thank you for your assistance.

[PEACHTTT]

Hello,have you met the problem about the datasets path？I totally follwed the README,but it seems I need other settings... I'm sure my datasets is in /home/name/vlfm/objectnav/hm3d/v1/val, and I run the vlfm.run in the directory "vlfm"

ERROR: File "/home/tcy/anaconda3/envs/vlfm/lib/python3.9/site-packages/habitat/datasets/pointnav/pointnav_dataset.py", line 51, in get_scenes_to_load raise FileNotFoundError( FileNotFoundError: Could not find dataset file data/datasets/objectnav/hm3d/v1/val

[Zeying-Gong]

I have not met this problem, the Readme.md file also introduces the process of Downloading the HM3D dataset. Actually, for habitat tasks, it requests both the scene dataset (like HM3D) and the task episode dataset (like objectnav episodes of HM3D).

You may need to check if there is a corresponding file for the error path or find more detailed instructions in Habitat's description of the datasets.

[PEACHTTT]

Thanks! I will try to check these terms.

[PEACHTTT]

I have not met this problem, the Readme.md file also introduces the process of Downloading the HM3D dataset. Actually, for habitat tasks, it requests both the scene dataset (like HM3D) and the task episode dataset (like objectnav episodes of HM3D).

You may need to check if there is a corresponding file for the error path or find more detailed instructions in Habitat's description of the datasets.

Sorry to bother again,does the environment you set totally followed the READ.ME? Or there might be some preparations?

[Zeying-Gong]

I found that may caused by the allow_sliding = False option, turn it to True and it become normal ~