JamesPerlman / TurboNeRF

A render engine for NeRFs!
MIT License
312 stars 16 forks source link

writing rendering images in separate thread #13

Closed MrNeRF closed 1 year ago

MrNeRF commented 1 year ago

Here is an suggestion to write the output images to disk in a separate thread. The whole process output seems to me smoother so writing to disk should not block anymore.

I am not sure about the -lpthread flag. On linux, it is required to compile the dependency. I am not sure but the file ending linux/windows stuff seems to mess up some lines of code that I did not touch or there is some formating in vs code working in the background.

Please, just check it out and let me know, if this is something you like or if I should revise it or abandon.

JamesPerlman commented 1 year ago

I like this idea, but I'm not too keen on passing a thread around like this. I believe this functionality should be left up to the implementation of the process controlling the renderer, not the buffer & utility functions. Check out this PR, does this do the same thing as what you want? https://github.com/JamesPerlman/NeRFRenderCore/pull/14

Further, this implementation only really speeds up the actual writing of the image to disk. Since the renderer.submit() and renderer.write_to() methods are both blocking, this doesn't speed the code up all that much in my tests.

For true parallel rendering that doesn't block training, we would have to implement some sort of synchronization where we copy the training params over to a separate buffer (to avoid having the renderer read from the same NN param buffers that the trainer/optimizer is writing to).

This is definitely a great idea. Check out PR-14 and lmk if this is sufficient for now. I didn't make the same CMakeLists.txt changes to allow for pthread, btw.

JamesPerlman commented 1 year ago

Actually, regarding parallel training/rendering, I am going to have to implement this soon anyway, since I want to be able to train & render in realtime inside of Blender. So that is on my roadmap very soon.

MrNeRF commented 1 year ago

Nice. I also notified that renderer.submit() and renderer.write_to() are the more severe bottlenecks. This writing to disk is not a huge performance win. But writing to disk is generally slow. So, it speeds things up just a little bit.

For true parallel rendering that doesn't block training, we would have to implement some sort of synchronization where we copy the training params over to a separate buffer (to avoid having the renderer read from the same NN param buffers that the trainer/optimizer is writing to).

This is definitely great and it would speed thinks up quite a bit.