search

YicongHong / Recurrent-VLN-BERT

Code of the CVPR 2021 Oral paper: A Recurrent Vision-and-Language BERT for Navigation
Other
150 stars 29 forks source link

Unable to test code #4

Closed gmuraleekrishna closed 3 years ago

gmuraleekrishna commented 3 years ago

Hello Yicong,

Can you please add a section in the README about using Matterport3DSimulator docker image with your code? The documentation is missing details on where to put the ResNet zip, prevalent JSON, and the PyTorch model. It is unclear how MatterPort3DSimulator works with your code.

Thanks

YicongHong commented 3 years ago

Hello Muraleekrishna,

Sorry, we are not going to work out a Docker image for our code. In fact, the code structure is very similar to many previous published VLN codebases.

For your specific questions: (1) ResNet Zip should be extracted in img_features, (2) Prevalent data files go into data, (3) the trained Pytorch models go into snap, (4) you can install the PyTorch.Transformer package and the Matterport3D.Simulator anywhere, and you should be able to run the code straightforward.

Edit: One more comment, this code uses the v0.1 version of the Matterport3D.Simulator. However, it should be quite easy to work with the latest version with some slight modifications.

Hope this helps. Cheers

ucasligang commented 3 years ago

Hi Yicong， I meet trouble of 'ModuleNotFoundError: No module named 'MatterSim''. Whether this is because I didn't successfully install Matterport3D Simulator? Besides, do I have to download the Matterport3D dataset to successfully install Matterport3D Simulator? I found that download the Matterport3D dataset is a little complex because it isn't easy to access. Looking forward to your help. Thanks.

YicongHong commented 3 years ago

Hi Ligang,

To import the MP3D simulator that you built, you might need to specify the path to the .so file, e.g. you can simply import sys sys.path.append('/path_to_the_Matterport_Simulator/build/') before importing MatterSim.

You do not need to download the MP3D RGB data unless you want to do visualization or actually walk in the rendered environments.

Cheers

ucasligang commented 3 years ago

Thank you for your reply. I will give it a try.

ucasligang commented 3 years ago

Hi Yicong,

I have another question when I run the test script 'bash run/test_agent.bash', there is an error of "MatterSim.Simulator' object has no attribute 'init'". I don't know whether you have an idea about it or not.

Looking forward to your reply. Thanks.

YicongHong commented 3 years ago

Hi Ligang,

As mentioned in ReadMe, this code uses the old version (v0.1) of the simulator, you might have installed the latest version. But it shouldn't be difficult for you to change some code to make them compatible with the latest version.

Hope this helps.

YicongHong commented 3 years ago

Hi Yicong,

I have another question when I run the test script 'bash run/test_agent.bash', there is an error of "MatterSim.Simulator' object has no attribute 'init'". I don't know whether you have an idea about it or not.

Looking forward to your reply. Thanks.

But still, you might want to install the old version because most of the published works on VLN are developed on the old simulator.