jacobkrantz / VLN-CE

Vision-and-Language Navigation in Continuous Environments using Habitat
https://jacobkrantz.github.io/vlnce/
MIT License
290 stars 56 forks source link

[RxR-Habitat] How to export a robot navigation video based on a baseline #53

Open 2201957 opened 1 year ago

2201957 commented 1 year ago

demo example2 Could you provide me with the code in the image above "Generate a video of a robot navigating with natural language guidance based on baseline"?

jacobkrantz commented 1 year ago

You can use config options to generate a video during evaluation: https://github.com/jacobkrantz/VLN-CE/blob/2375f186cd8a1224a5ff15efa4c0ac05edfb2d42/vlnce_baselines/config/default.py#L23-L24

If you want to update the way in which frames are produced, you can modify/replace the functions in https://github.com/jacobkrantz/VLN-CE/blob/master/habitat_extensions/utils.py

Custom elements in the above videos (arrows, additional text) were added using opencv utilities.

chalindunis commented 12 months ago

Hi @jacobkrantz what can be reason for not getting a robot navigating video even after make changes to the [default.py](https://github.com/jacobkrantz/VLN-CE/blob/2375f186cd8a1224a5ff15efa4c0ac05edfb2d42/vlnce_baselines/config/default.py#L23-L24) as follows:

_C.VIDEO_OPTION = ["disk"]  # options: "disk", "tensorboard" 
 _C.VIDEO_DIR = "data/videos/debug" 

Do we need to make any others changes? One more question, what's the difference between using 'disk' and 'tensorboard' and how can they be accessed?

chalindunis commented 11 months ago

Hi @jacobkrantz what can be reason for not getting a robot navigating video even after make changes to the [default.py](https://github.com/jacobkrantz/VLN-CE/blob/2375f186cd8a1224a5ff15efa4c0ac05edfb2d42/vlnce_baselines/config/default.py#L23-L24) as follows:

_C.VIDEO_OPTION = ["disk"]  # options: "disk", "tensorboard" 
 _C.VIDEO_DIR = "data/videos/debug" 

Do we need to make any others changes? One more question, what's the difference between using 'disk' and 'tensorboard' and how can they be accessed?

Solved it. Thank you!

MyShepherd666hzb commented 3 months ago

嗨,即使在更改 [default.py](vlnce_baselines/config/default.py) 后仍无法获取机器人导航视频的原因是什么:

_C.VIDEO_OPTION = ["disk"]  # options: "disk", "tensorboard" 
 _C.VIDEO_DIR = "data/videos/debug" 

我们是否需要进行任何其他更改?还有一个问题,使用“disk”和“tensorboard”有什么区别,如何访问它们?

解决了它。谢谢!

How did you solve this problem, and what changes are still needed? I really want to reproduce