How to train the navigation policy

[yhykid]

Hi Simar!Your work is very great!! When I tried to reproduce the code of "VinL",I can finish the locomotion part,including training and test. But when I tried to use the navigation policy trained by myself,following the instruction in https://github.com/joannetruong/habitat-lab/blob/kin2dyn/README.md,it does not work.The report is followed: RuntimeError: Error(s) in loading state_dict for PointNavResNetPolicy: size mismatch for net.prev_action_embedding.weight: copying a param with shape torch.Size([32, 3]) from checkpoint, the shape in current model is torch.Size([32, 2]). size mismatch for net.tgt_embeding.weight: copying a param with shape torch.Size([32, 4]) from checkpoint, the shape in current model is torch.Size([32, 3]). size mismatch for net.visual_encoder.compression.0.weight: copying a param with shape torch.Size([102, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([137, 256, 3, 3]). size mismatch for net.visual_encoder.compression.1.weight: copying a param with shape torch.Size([102]) from checkpoint, the shape in current model is torch.Size([137]). size mismatch for net.visual_encoder.compression.1.bias: copying a param with shape torch.Size([102]) from checkpoint, the shape in current model is torch.Size([137]). size mismatch for net.visual_fc.1.weight: copying a param with shape torch.Size([512, 2040]) from checkpoint, the shape in current model is torch.Size([512, 2055]). size mismatch for action_distribution.mu.weight: copying a param with shape torch.Size([3, 512]) from checkpoint, the shape in current model is torch.Size([2, 512]). size mismatch for action_distribution.mu.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]). size mismatch for action_distribution.std.weight: copying a param with shape torch.Size([3, 512]) from checkpoint, the shape in current model is torch.Size([2, 512]). size mismatch for action_distribution.std.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]). And I dont know how to fix it.So how can I train the navigation used in VinL??? Thanks!!!

[SimarKareer]

Hi, apologies for such a late response I oddly didn't get notified! Thanks so much for your interest in our work! We've moved some code around recently

• Use this repo to train the navigation agent (instructions in the readme). You'll need to swap out the aliengo URDF, for whatever robot you're using.
• Install this version of habitat lab (already included in the ViNL repo as a submodule)

[yhykid]

Hi,Simar.Thanks for your help,but I still have some question.I followed your instruction to train the navigation agent,but which config file should I use?Is it "ver_hm3d_robot_nav.yaml"?

[SimarKareer]

Hi @yhykid! Yes, use "ver_hm3d_robot_nav.yaml". We also remembered that we need to randomize the rotation of the depth camera while training the nav policy, so we've included that change in this verison of habitat lab. So use this habitat lab instead (I'll update the readme)

[SimarKareer]

Please let me know if you have any more issues, if not you can close this thread!

[yhykid]

Thanks for your help!But I still have problems.I cant run the command"pip install habitat-baseline",and when I run the command"python -um robot_nav.run --config-path=../ --config-name=ver_hm3d_robot_nav",the error is coming: Traceback (most recent call last): File "/home/yhy/mambaforge/envs/robot_nav/lib/python3.9/runpy.py", line 188, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, _Error) File "/home/yhy/mambaforge/envs/robot_nav/lib/python3.9/runpy.py", line 111, in _get_module_details __import__(pkg_name) File "/home/yhy/code/vinl/robot_nav/robot-nav-main/robot_nav/__init__.py", line 2, in <module> import robot_nav.auto_stop_success File "/home/yhy/code/vinl/robot_nav/robot-nav-main/robot_nav/auto_stop_success.py", line 4, in <module> from habitat.config.default_structured_configs import SuccessMeasurementConfig ModuleNotFoundError: No module named 'habitat.config.default_structured_configs'

I really need your help!!!It is really hard for me to reproduce! Thank you !!!

[SimarKareer]

Just to be sure, did you run "pip install habitat-baseline" or "pip install -e habitat-baselines". Also did you clone the version of habitat-lab that I linked in the post above? If you could create a new conda env, and paste all the commands you run here, I can try and reproduce the issue you're having

[yhykid]

Thank you,Simar!!! I follow your instruction again,and the file is the commands I run with its output. Thank you again!! output_command.txt

[SimarKareer]

Sorry for such a delay, were you able to install habitat? @yhykid

[yhykid]

sorry,I cant install habitat because the file ' setup.py ' may be incorrect

[yhykid]

Simar,will you go to ICRA2024 yokohama?

[SimarKareer]

Sorry for the delay, I didn't attend ICRA2024 sadly! Were you able to resolve this?