JuliaDiff / SparseDiffTools.jl

Fast jacobian computation through sparsity exploitation and matrix coloring
MIT License
237 stars 41 forks source link

Forward over reverse Hessian methods #27

Open ChrisRackauckas opened 5 years ago

ChrisRackauckas commented 5 years ago

An implementation of forward-over-reverse, i.e. applying forward mode autodiff to the result of reverse autodiff, can be a way to exploit AD for the computation of Hessians. It might make sense to directly implement this into Zygote.

gdalle commented 1 week ago

In DifferentiationInterface, if you compute a sparse Hessian with SecondOrder(AutoForwardDiff(), AutoZygote()), this is exacly what will happen. And since Zygote uses ForwardDiff under the hood for its dense Hessian, I took the liberty to do the same for HVPs, so you can even use AutoZygote() alone and it will work