Open ChrisRackauckas opened 5 years ago
In DifferentiationInterface, if you compute a sparse Hessian with SecondOrder(AutoForwardDiff(), AutoZygote())
, this is exacly what will happen.
And since Zygote uses ForwardDiff under the hood for its dense Hessian, I took the liberty to do the same for HVPs, so you can even use AutoZygote()
alone and it will work
An implementation of forward-over-reverse, i.e. applying forward mode autodiff to the result of reverse autodiff, can be a way to exploit AD for the computation of Hessians. It might make sense to directly implement this into Zygote.