search

ashawkey / torch-ngp

A pytorch CUDA extension implementation of instant-ngp (sdf and nerf), with a GUI.
MIT License
2.08k stars 273 forks source link

Mip-NeRF #82

Open JonathanLehner opened 2 years ago

JonathanLehner commented 2 years ago

What do you think about Mip-NeRF and Mip-NeRF 360? Do you have ideas how to integrate them?

ashawkey commented 2 years ago

@JonathanLehner Hi, The ray sampling and integrated position encoding in mip-nerf seems to require substantial changes to the current ray sampling implementation. How to combine it with hashgrid encoding is also a problem. For the ray contraction in mip-nerf 360, we may be able to implement a similar approach using the L-inf norm instead of the L2 norm (so the sphere becomes a box to suit the hashgrid encoding) like in DVGO.

jingyibo123 commented 2 years ago

The ray sampling and integrated position encoding in mip-nerf seems to require substantial changes to the current ray sampling implementation.

Combining the mip-nerf sampling strategy should lead to major slowdown of training and inference speed, right ?

Also I'm very curious on how the "distortion loss" will perform with ngp.

Correct me if I'm mistaken.

JonathanLehner commented 2 years ago

What about Ref-NeRF? it might be easier

JonathanLehner commented 2 years ago

https://dorverbin.github.io/refnerf/