WASM Module?

[DennisSmolek]

I am a lowly web developer but with the rise of JS capabilities myself and others are trying to bring some native level functionality to the web.

I saw your recent article about C++ and WASM and figured a simple package like this would be a great use-case for that.

Before bashing my own head against the wall trying to figure it out I wanted to see if it's something someone more experienced would like to do.

The ideal situation would be a WASM module with methods exposed to either send a file or image buffer in and get a denoised result out.

The use cases would be three-gpu-pathtracer where using this denoiser has been discussed (but was going to pull out the original models and use in a web based DNN)

Or in my case a volume renderer to reduce the number of rays needed etc.

This tweet actually talks about someone who uses tensorflow.js and the core models to use OIDN on the web.

Not sure which is the best approach...

[Twinklebear]

That tweet thread is super cool! I think to run the denoising on the GPU, the best approach would be to reproduce the OIDN network with Tensorflow.js or ONNX and run it with the respective library's WebGPU backend, essentially what's described in those tweets. The OIDN weights are open source, so just setting up the network architecture and loading in the weights should work (which sounds like what Yi Shen did). That could be packaged up into a library that could take a GPU texture or image data on the CPU to denoise.

If you wanted to run the denoising on the CPU it could be done by compiling the CPU version of OIDN to WASM + WASM SIMD for ISPC, but if I remember right OIDN also uses multithreading to run the denoising on the CPU w/ TBB, that may be challenging to compile to WASM. Running on the GPU is probably best for perf and reducing memory transfer costs too.

In either case though I don't think the oidn-rs crate would be of much use, since all this library does is wrap the OIDN C++ library to setup bindings for Rust. So while the Rust bindings would probably be easy to compile to WASM, the real challenge would be in compiling the OIDN library for WASM.

[DennisSmolek]

Thanks for the feedback!

I'm going to go ahead and take a stab at exporting the weights with Tensorflow.js myself. With the webGL and WebGPU backends it's surprisingly fast, so I'm hoping to get some decent results. The WebGPU backend also persists the buffer on the GPU meaning I could call it directly from a fragment shader as well so that's interesting...

Getting OIDN to run was a lesson in humility... And yes OIDN uses TBB, which apparently has pretty good WASM support although it's beyond my skillset to get OIDN's implementation to play nice with everything.

I was thinking the Rust package for a WASM module because my general comfort in rust over C++ and (what at least seems) easier support for WASM generation.

Thanks again for the insights, I'll keep you in the loop if I pull it off!