search

Tangshitao / MVDiffusion

MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion, NeurIPS 2023 (spotlight)
501 stars 27 forks source link

Some confusion about details of homography matrix. #7

Open Estrellama opened 1 year ago

Estrellama commented 1 year ago

Hello, I really appreciate your work, but there are some issues regarding the details.

I'm confused about how the homography matrix is calculated in the panorama image generation task of your work. What does the phrase "pixel-to-pixel correspondences are available, such as perspective crops from panorama" mean in the abstract section of the paper? Are you saying that all adjacent image pairs in the panoramic image share the same homography matrix? As far as I know, the homography matrix for each image pair needs to be recalculated.

Thank you for your time and for considering my questions. Looking forward to your response. Best regards, Enhui Ma

Tangshitao commented 1 year ago

We set a fixed homography matrix whose rotation is 45 degrees. The FOV of each image is 90 degrees, so 8 images can form a 360-degree view.

Estrellama commented 1 year ago

Thank you for your fast reply.

In order to better understand the details of matrix computation, I calculated the homography matrix below according to your suggestions. I would like to confirm with you if my approach is correct.

Details of my approach: I set up 4 sets of corresponding points as shown in the figure below to calculate the homography matrix. The effect of applying homography transformation to the image before and after is shown in the two sub-figures. image Thank you for considering my confusion.

Tangshitao commented 1 year ago

Yes, that's correct.

Estrellama commented 1 year ago

Thanks for you fast reply again! Best regards.