Tangshitao / MVDiffusion

MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion, NeurIPS 2023 (spotlight)
507 stars 27 forks source link

Confusion about the calculation of the homography matrix. #20

Open Estrellama opened 1 year ago

Estrellama commented 1 year ago

Dear Tang,

I would like to express my appreciation for your outstanding work. I have some further questions regarding the calculation of the homography matrix.

I noticed that in your calculation of pixel correspondence, you only considered the rotation angle of the camera and did not take into account translation (meaning that your camera extrinsics only include a rotation matrix). I understand that this is related to the specific setting of panoramic image generation.

So, my question is, if a camera undergoes both translation and rotation simultaneously (meaning that the camera extrinsics should include both a translation matrix and a rotation matrix), does your approach for computing the homography matrix still apply in this situation? In other words, is it because it does not work in such a scenario that your experiments were limited to generating only panoramic images?

Thank you, and I look forward to your response! Best regards.

kkaiwwana commented 1 year ago

This answer would help: https://stackoverflow.com/questions/63232589/compute-homography-matrix-based-on-intrinsic-and-extrinsic-camera-parameters

Tangshitao commented 1 year ago

Sorry about delayed answers. Our work can only generate panoramic images.