search

avhz / RustQuant

Rust library for quantitative finance.
https://avhz.github.io
Apache License 2.0
1.14k stars 133 forks source link

`autodiff`: add support for Jacobian and Hessian matrices. #87

Open avhz opened 1 year ago

avhz commented 1 year ago

Currently only gradients can be computed via the RustQuant::autodiff module. Adding support for full Jacobians and also higher-order derivatives like the Hessian would be nice.

Xblow commented 1 year ago

@avhz Hi, I am not familiar with the codebase but I will try to figure something out in the next couple of days. Any specific reason why this issue is considered difficult?

avhz commented 1 year ago

Jacobian matrices shouldn't be too bad to implement (just a collection of gradients) but I have not figured out higher order derivatives. From what I've read it's a lot more work.

Xblow commented 1 year ago

I have looked through and I am interested in tackling this issue, but will need a bit of time, since I just recently started playing with Rust and have plenty of knowledge gaps to fill...

  1. I have found a description of an autodiff method you are implementing. Is your implementation from scratch or is based on something else? If you have some specific resources, please mention them, that would be very helpful.
  2. It looks like the Jacobian can be realised with the existing .accumulate method on VariableArray instance. I was not able to verify that it works yet though.
  3. Seems that Hessian should be safely done as a double pass of accumulate, but I haven't thought of how to implement that in practice yet.
avhz commented 1 year ago

The two best references I know of are:

The VariableArray object is really a WIP and does not actually work as intended, since we can't fill ndarrays (nor nalgebra matrices/vectors) with the Variable type. Applying accumulate to a vector output function should be all that needs to happen, just with a nice/intuitive API.

The last point about the Hessian is what would be ideal, but since the current methods return f64s not Variables, we can't apply it twice. For Hessian accumulation, there are 3 modes: forward-over-reverse, reverse- over-forward, and reverse-over-reverse. We would have to do the last, since forward is not implemented.