peteanderson80 / Matterport3DSimulator

AI Research Platform for Reinforcement Learning from Real Panoramic Images.
Other
472 stars 128 forks source link

The panoramic view is not stiched well (first and last image are looking up or down with non-zero elevation)! #110

Open lukewenMX opened 2 years ago

lukewenMX commented 2 years ago

The generated panoramic view from the skybox images is not stiched well by runing the downsize_skybox.py, that is the _skybox_small.jpg is not an accurate panoramic view especially for the first and last images (one is looking up and one is looking down with an non-zero elevation) . An instance has been shown in below, please give some comments and suggestions, thanks. 0b22fa63d0f54a529c525afbf2e8bb25_skybox_small

peteanderson80 commented 2 years ago

This is working as intended. The image is a skybox / cubemap not an equirectangular panorama, so it's actually 6 different images representing the faces of a cube and they are just joined together for convenience (it speeds up file loading to have fewer and larger files).

lukewenMX commented 2 years ago

@peteanderson80 Hi, Peter. Thanks for your sharp reply and the answer really helps me. By the way, do you have any ideas about how to generate the 'equirectangular panorama' shown as perfectly stiched together? I did not find any program which supports to load the well-trained agent to interact with Matterport 3D simulator and at the meantime showing the "equirectangular panorama" . Will appreciate your further answers and any suggestions. Thank again.

lukewenMX commented 2 years ago

@peteanderson80 Hi, Peter. I think I just got the idea of the skybox, and the question about the equirectangular panoramaļ¼Œplease just ignore. Many thanks.

lukewenMX commented 2 years ago

By the way, is there any convenient way to show the traversed path by the agent from the top-down view of the simulated house, as dipicted below (Picture from the published paper reference: Li, Xiujun, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, and Yejin Choi. "Robust navigation with language pretraining and stochastic sampling." arXiv preprint arXiv:1909.02244 (2019).): path visualization from TOP Many thanks in advance for any help and suggestions.

jdiazram commented 1 month ago

Hi @lukewenMX, could you solve the creation of an equirectangular panorama from skyboxes? I used py360convert, but to analyze the result, py360convert is based on: image1

Then, after analyzing the original paper, "Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments," on page 11, I think the heading is another. Like this:

image2