search

dcharatan / pixelsplat

[CVPR 2024 Oral, Best Paper Runner-Up] Code for "pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction" by David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann
http://davidcharatan.com/pixelsplat/
MIT License
830 stars 56 forks source link

question about ray projection #59

Closed beautifulchoi closed 5 months ago

beautifulchoi commented 5 months ago

Hello, thanks for your great work! it's really awesome !! btw while I'm on studying your code line by line, I can't understand this comment

    # Compute the ray's projection at zero depth. If an origin's depth (z value) is
    # within epsilon of zero, this can mean one of two things:
    # 1. The origin is at the camera's position. In this case, use the direction
    #    instead (the ray is probably coming from the camera).
    # 2. The origin isn't at the camera's position, and randomly happens to be on
    #    the plane at zero depth. In this case, its projection is outside the image
    #    plane, and is thus marked as invalid.

it's inside the src.geometry.epipolar_lines.py at 186 line. what I think is , it seems first case is that two camera's center is same, but other one is two camera center is not same and parallel so it can not be projected. is it correct...?

dcharatan commented 5 months ago

As you say, the first case happens when two cameras share the same origin. The second case happens when the cameras don't share the same origin, but the second camera is at $z = 0$ in the first camera's coordinate frame. This is a problem because projecting a point at $z = 0$ is numerically unstable. It would be possible to fix this (e.g., by stepping along the ray that's being projected to pick a new origin), but since this case is pretty rare, we didn't bother. Comment 2 is actually wrong, since the projection could still be within the image plane.