search

NVlabs / instant-ngp

Instant neural graphics primitives: lightning fast NeRF and more
https://nvlabs.github.io/instant-ngp
Other
15.57k stars 1.89k forks source link

Python API - Are the following functionalities exposed? #1504

Open dalnoguer opened 5 months ago

dalnoguer commented 5 months ago

Hello!

I am using the python API and saw that by changing the ERenderMode from Shade to Depth it is possible to render either the color frame or the depth.

However, I was wondering if it is possible to render depth distance and image in one pass?

Also, is it possible to query the nerf field from the python API? I would like to pass (x, y, z, theta, phi) and get the resulting rgb and density.

Thanks a lot!

Tom94 commented 5 months ago

Hi Dal, INGP already renders a depth buffer alongside the "main" rendering (as determined by ERenderMode). There's no Python binding for this, but you could for example modify the existing Python binding for RGBA rendering (render_to_cpu(...)) to also copy the rendered depth buffer (I think m_windowless_render_surface.depth_buffer()) into a CPU array and to return that.

The other functionality (passing x, y, z, and direction explicitly) is also not exposed. You'd have write your own call to the NeRF model and pipe its result to Python, similar to the existing bindings.