VladimirYugay / Gaussian-SLAM

Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting
https://vladimiryugay.github.io/gaussian_slam
MIT License
852 stars 50 forks source link

Camera frustum planes computation bug? #22

Closed AmineElhafsi closed 3 months ago

AmineElhafsi commented 3 months ago

Hello,

I was looking at this portion for the code and it seems like the cross product computation for the top and bottom planes might be off.

Shouldn't the last two cross products look as follows?

...

# top
torch.cross(
    frustum_corners[5] - frustum_corners[1], # top right far - top right near
    frustum_corners[0] - frustum_corners[1] # top left near - top right near
),
# bottom
torch.cross(
    frustum_corners[6] - frustum_corners[2], # bot left far - bot left near
    frustum_corners[3] - frustum_corners[2] # bot right near - bot left near 
)

...

I'm assuming the frustum corners correspond to the following (in pixel+depth coordinates which are transformed to world coordinates before computing the planes):

corners = np.array(
        [
            [0, 0, min_depth], # 0 - top left near
            [width, 0, min_depth], # 1 - top right near
            [0, height, min_depth], # 2 - bottom left near
            [width, height, min_depth], # 3 - bottom right near
            [0, 0, max_depth], # 4 - top left far
            [width, 0, max_depth], # 5 - top right far
            [0, height, max_depth], # 6 - bottom left far
            [width, height, max_depth], # 7 - bottom right far
        ]
    )

I'm just checking but any clarification would be much appreciated!

unique1i commented 3 months ago

Hi @AmineElhafsi,

thanks for the spot, this was a bug. The code didn't include top and bottom planes and led to more points selected. Since this part is preparing for neighbor search, I expect it has minimum effect on the pipeline.

AmineElhafsi commented 3 months ago

Got it - thanks!