Question about panorama generation at inference

[lukasHoel]

Thanks for sharing this interesting work! :) After reading about panorama generation in the paper, I have the following question:

How is the correspondence t^l between the frames (Fig. 3) obtained at inference/test-time? Is it given as an input in every denoising step?

Does this mean you do the following things at inference/test-time?

1. Take any existing panorama and project to 8 images
2. Compute correspondences between the 8 images
3. Use the correspondences to generate 8 new images starting from random noise with the generation module

Thank you for clarifying!

[Tangshitao]

The correspondences are the same for all the panorama. See the panorama projection matrix here, https://www.cambridgeincolour.com/tutorials/image-projections.htm

[lukasHoel]

I see, thanks for the link, now I understand the usage of a fixed homography matrix. May I ask a follow-up question: did you see limitations in fixing the correspondences, e.g. using always 8 frames with the same amount of overlap? Concretely, does it limit the layouts that can be generated in any way? Thanks again :)

[Tangshitao]

Can you clarify what layout is? Is it a global structure of room?

[lukasHoel]

Yes

[Tangshitao]

I didn't see any limitation of global structure. You can try our demo.