API for high-order scalar derivatives

[jrevels]

It'd be nice to use the existing DiffBase.DiffResult API to get higher-order scalar derivatives via derivative!.

For example, it would be cool if you could do the following:

julia> using ForwardDiff: derivative!

julia> using DiffBase: DiffResult

julia> derivative!(DiffResult(0., 0., 0.), sin, 1.0)
DiffBase.DiffResult{2,Float64,Tuple{Float64,Float64}}(0.8414709848078965,(0.5403023058681398,-0.8414709848078965))

Currently, derivative! will just calculate the first derivative:

julia> derivative!(DiffResult(0., 0., 0.), sin, 1.0)
DiffBase.DiffResult{2,Float64,Tuple{Float64,Float64}}(0.8414709848078965,(0.5403023058681398,0.0))

cc @YingboMa

[papamarkou]

@jrevels, is this issue part of your goals? It would be nice to add this functionality for, say, second order scalar derivatives.

[DNF2]

Is there any chance for doing this? There is an interface for calling derivative! on a DiffResult with space for higher derivatives, and also for retrieving the higher derivatives from DiffResult. But they are just ignored by derivative!:

julia> g = x->3x^3+2x^2-2x+11;

julia> res = DiffResults.DiffResult(NaN, (NaN, NaN, NaN));

julia> res = ForwardDiff.derivative!(res, g, 1.0)
ImmutableDiffResult(14.0, (11.0, NaN, NaN))

julia> DiffResults.value(res)
14.0  # 😃 

julia> DiffResults.derivative(res, Val{1})
11.0  # 😃 

julia> DiffResults.derivative(res, Val{2})
NaN  # 😢 

Is there any alternative way to calculate a value, first derivative, and a second derivative without doing redundant work, like there is using DiffResults.GradientResult, DiffResults.HessianResult, etc.?

[gerlero]

@DNF2 AbstractDifferentiation.jl should be able to do it (since https://github.com/JuliaDiff/AbstractDifferentiation.jl/pull/122).

I also have an open PR (https://github.com/JuliaDiff/ForwardDiff.jl/pull/678) where I add this functionality to this package, but it hasn't gotten anywhere yet.

[DNF2]

Thanks for the heads-up, @gerlero. Does that mean this works with some other backends, but not currently with ForwardDiff.jl?

[gerlero]

@DNF2 It works with the ForwardDiff backend, but you need to install AbstractDifferentiation package and use its interface (you cannot get that functionality with ForwardDiff alone).