[Question] Does/could DrJit support Jacobians?

[lkskstlr]

Hi @wjakob,

thank you for putting out DrJit, it's a really cool framework!

I have been thinking if DrJit supports Jacobian computation in the context of SfM/SLAM, maybe with some additional coding required. The Jacobian J would be of size n_residuals x n_params and (block)-sparse. A residual could either be a point-correspondence from feature matches, e.g. ORB/SIFT, or a photometric residual. Usually, the Jacobian isn't instantiated directly, but either one computes the (Schur complement) of J^T J or JVPs, both in the context of a Gauss-Newton type solver.

I know that you have a somewhat related paper: Large Steps in Inverse Rendering of Geometry (https://rgl.s3.eu-central-1.amazonaws.com/media/papers/Nicolet2021Large.pdf), though I am not sure if the trick would be transferable.

A possible application would be PERF: Performant, Explicit Radiance Fields (https://arxiv.org/abs/2112.05598) just to give a concrete instantiation of the idea.

Bests Lukas

[wjakob]

While interesting, this is beyond the scope of Dr.Jit that is more focused on the computational needs of differentiable rendering. Something like this may be more relevant for your needs: https://boris.unibe.ch/173009/1/v41i5pp113-124.pdf

[lkskstlr]

Thanks for the interesting paper!