SLANG.D as differentiable renderer for Warp

NVIDIA / warp

A Python framework for high performance GPU simulation and graphics

https://nvidia.github.io/warp/

Other

3.58k stars 201 forks source link

SLANG.D as differentiable renderer for Warp #151

Open korzen opened 8 months ago

korzen commented 8 months ago

Hello,

I have just come across https://research.nvidia.com/labs/rtr/publication/bangaru2023slangd/ and it instantly got me thinking whether SLANG.D could be used as a lightweight differentiable rasterizer or raytracer (RT-Cores accelerated?) for simulation in Warp? Some example integration would be great. Cheers!

mmacklin commented 7 months ago

Hi @korzen, agreed, I think that it can totally make sense to do diff. render in something like SLANG.D and physical simulation in Warp. We do have an example of a differentiable raytracer in Warp here: https://github.com/NVIDIA/warp/blob/main/examples/example_diffray.py, but will look into how we might integrate with SLANG as well.

Best, Miles

jprnd commented 5 months ago

could this be done with the Kaolin Library https://kaolin.readthedocs.io/? Are there advantages to SLANG.D over Kaolin as the diff renderer?@mmacklin any advice on interop? Can I just keep everything in USD and stream from warp to Kaolin?