search

BachiLi / redner

Differentiable rendering without approximation.
https://people.csail.mit.edu/tzumao/diffrt/
MIT License
1.38k stars 139 forks source link

Procedural geometry #123

Open nkyriazis opened 4 years ago

nkyriazis commented 4 years ago

Similarly to #118 , and perhaps more easily accommodated, it would be nice if the geometry could be sampled procedurally, instead of having to be instantiated in a triangular mesh. Consider the example of trying to redner (yes, it's a verb now :smile:) a perfect sphere.

BachiLi commented 4 years ago

I agree this would be very useful but this would be a research project by itself. This is something I've been thinking for a while. Will keep you posted if I or other people have made progress on this.

nkyriazis commented 4 years ago

:open_mouth: Wow, I didn't know. Any comment on why this is so challenging?

My line of thought was similar to #118. At some point, rays are queried for intersection with geometry. Optix does allow for procedural geometry, but, understandably, this flexibility is not that easy or straightforward to expose to the user. However, in the presence of procedural geometry, the intersection result can be the combination of two procedures: 1) Optix/Embree, 2) Custom query of geometry provider. Each ray should get the minimum distance intersection of the 2 results. Am I even half-right there? :stuck_out_tongue:

BachiLi commented 4 years ago

That's only part of the problem. The remaining questions are: 1) extracting boundary/edges for derivative sampling 2) differentiating the intersection code 3) differentiability if we allow recursive geometry construction (fractals are not differentiable by default) 4) Interface with PyTorch/TensorFlow

nkyriazis commented 4 years ago

Thanks for the insights!

My understanding is that the main issue is differentiability in and beyond boundaries. What if the user were to supply the geometry as a signed distance function of the boundary? It might not be a panacea but it might be a good supplement.