nerfstudio-project / nerfacc

A General NeRF Acceleration Toolbox in PyTorch.
https://www.nerfacc.com/
Other
1.37k stars 113 forks source link

Filtering strategy (instead of iterating ray samples) for traverse_grids? #269

Closed Yosshi999 closed 4 months ago

Yosshi999 commented 6 months ago

Currently traverse_grids samples points by proceeding the ray little by little, but in this strategy the sampled points are sometimes located at a vacuum voxel (I mean the voxel where binaries[voxel_id] is False) because of float errors. I think you are aware of this according to the comment here.

In contrast to this, we can avoid the error when we sample all points on a ray and then filter points where binaries[voxel_id] is False or outside of the aabb. Do you think this will cause a performance drop or other problems? I'm interested in implementing this strategy if it is reasonable.

liruilong940607 commented 6 months ago

This is likely to bring a bit performance drop but more important would lead to more memory usage because you would need to allocate memory for all samples -- and this can be pretty wasteful of memory usage if the space is sparse enough.

In practice, however, network would be the bottleneck for both speed and memory allocation so I would expect not too much of difference between the two