search

dcharatan / pixelsplat

[CVPR 2024 Oral, Best Paper Runner-Up] Code for "pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction" by David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann
http://davidcharatan.com/pixelsplat/
MIT License
830 stars 56 forks source link

Adjust camera intrinsics when resizing and cropping #58

Closed Trainingzy closed 5 months ago

Trainingzy commented 5 months ago

Hi, thanks a lot for your great work!

I have a question regarding the camera intrinsics. In your implementation of center crop, focal length is adjusted. I think focal length should be same when center crop while only adjusted when resizing. As the focal length is normalized, it should not be changed when resizing and center cropping.

If I understand it correctly, when center crop to square, the bugs are introduced here.

dcharatan commented 5 months ago

When center-cropping unnormalized intrinsics, you're right that the principal point should change, but the focal length shouldn't. Unnormalized intrinsics look like this:

[[fx,  0, cx],
 [ 0, fy, cy],
 [ 0,  0,  1]]

However, our intrinsics are normalized as follows:

[[fx / w,      0, cx / w],
 [     0, fy / h, cy / h],
 [     0,      0,      1]]

Once you apply the normalization, the math counter-intuitively makes the focal length change for center-cropping while the principal point stays the same. For example, the new top-left value (corresponding to fx) becomes (fx / old_w) * old_w / new_w, and the top-right value (corresponding to cx) becomes (cx / old_w) * old_w * (new_w / old_w) / new_w = cx / old_w, assuming that cx and cy are 0.5.

Trainingzy commented 5 months ago

Okay. Thanks a lot for answering!