nerfstudio-project / gsplat

CUDA accelerated rasterization of gaussian splatting
https://docs.gsplat.studio/
Apache License 2.0
2.09k stars 264 forks source link

Question about info["gaussian_ids"] in packed mode #450

Open wooseokS opened 4 days ago

wooseokS commented 4 days ago

Hi, thanks for your hard works.

I'm confusing exactly what info["gaussian_ids"] means when packed=True.

I thought that a key "gaussian_ids" means the IDs of individual 3D Gaussians, so that we can easily access on the attributes of Gaussians.

But when I ran this code right after the step_post_backward process,

if step % 50 == 0:
    ids = info["gaussian_ids"]
    print(f"{step} : {ids.size()}")
    print(f"{step} : {self.splats['means'].size()}")

It appears that the two tensor's sizes are different. (I expected ids.size(dim=0) == self.splats['means'].size(dim=0))

0 : torch.Size([24137])
0 : torch.Size([54275, 3])
50 : torch.Size([28588])
50 : torch.Size([54275, 3])
...

Could you explain me about what exactly the "gaussian_ids" means in packed mode?

I appreciate for your kindness.

Metro1998 commented 3 days ago

Do use the batch_size rasterization? It may cause info['gaussian_ids'] < info['means2D'], as the former just records which gaussian primitives have been rasterized in all batch cameras.

wooseokS commented 1 day ago

Oh, I see. So info["gaussian_ids"] means the id of gaussians which will be rendered in the batch cameras. Is it right?