search

kwea123 / ngp_pl

Instant-ngp in pytorch+cuda trained with pytorch-lightning (high quality with high speed, with only few lines of legible code)
MIT License
1.26k stars 156 forks source link

Asking for comments to ray-marcher implementation #24

Closed MultiPath closed 2 years ago

MultiPath commented 2 years ago

Hi, thank you very much again for this nice re-implementation of pytorch instant-ngp. I have been reading and playing with it in another codebase. However, I found it was very difficult to understand correctly the ray marcher implementation. https://github.com/kwea123/ngp_pl/blob/master/models/csrc/raymarching.cu For example, I could not understand terms like cascades, scale, exp_step_factor which are very nontypical compared to the original NeRF implementations. It would be nice to have more comments to explain how raymarcher works. Also, there is no hierarchical (importance) sampling in this case? Thanks very much!

kwea123 commented 2 years ago

Hi, the idea is in the paper, section E (page14). scale is the scale of the scene, for synthetic scenes we have scale=0.5 which means the scene is bounded in [-0.5, 0.5]. For custom data this can be larger to cover more space we keep a set of density grids to skip empty space and to sample points only near the surface, so there is no hierarchical sampling.

cascades is the number of density grids we have. The first density grid covers [-0.5, 0.5], the second [-1, 1], etc. So the number of cascades depend on scale.

exp_step_factor is called "exponential stepping" in the paper, it means instead of constant stepping (like in NeRF coarse sampling), we take larger steps when the sample points are farther from the camera (like constant step in NDC, it means larger steps when sample points are far in real space). exp_step_factor controls this.

I'll find some time to write more comments and with some graphs

MultiPath commented 2 years ago

That would be great! Thanks. So for cascades, does it mean that there are multiple dense grids for a single scene? I also need to read the original paper more carefully again. Did not notice these details.

kwea123 commented 2 years ago

Yes, there are multiple grids with growing spatial coverage. It would be more clear if I can show you a plot, but unfortunately I'm still debugging... now the synthetic data only have 1 density grid covering [-0.5, 0.5]

MultiPath commented 2 years ago

Thank you very much! This ray marching part looks quite complicated, especially when it was implemented in CUDA. Really appreciate your code (which is already more friendly to read compared to the original one IMO).

kwea123 commented 2 years ago

Custom data support is ready now! I will create a graph using the data to explain things more clearly

MultiPath commented 2 years ago

Great, looking forward to it!