Extract rays with sampling points and density values

[pellethielmann]

Discussed in https://github.com/NVlabs/instant-ngp/discussions/1058

^{Originally posted by **pellethielmann** November 8, 2022} Hi everyone, I want to get the rendered rays out of the NGP for NeRF. Means I want to have the Ray for each pixel of the rendered image with all sample points along this ray and the according values of the density function. In the original NeRF Code I think this was called depthmap or disparity map. The purpose is the following: Taking the maximum density value and the associated sampling point per ray and the 5D in combination with the sampling strategy and the total length of the ray, I should get an exact position of the volume and therefore a good depth value. Would appreciate any help solving my task or hints where to find the relevant parts in the code.

[jc211]

InstantNGP does not currently have a means of giving you the points and their densities. It only provides the result of the ray marching. However, its not so difficult to implement yourself. I've built something like this for a paper on epistemic uncertainty with NeRFS. The plot you see is the distribution of the densitites along one ray (This is an old image with a bug in the program so the calculated depth drawn is not correct). Storing the values of every ray in the image may give you memory constraint problems. The place you want to start looking is nerf tracer class. https://github.com/NVlabs/instant-ngp/blob/6c06f0bca6daed6d99f9e8844af65a1358463cd4/src/testbed_nerf.cu#L2063