Add sparse Hessian decompression

[ElOceanografo]

Following up on https://github.com/JuliaDiff/SparseDiffTools.jl/issues/142, this is a basic implementation of sparse Hessian decompression using matrix coloring. (See also https://github.com/SciML/Optimization.jl/issues/269.) Basic interface is modeled on the existing one for Jacobians; it includes forwarddiff_color_hessian and forwarddiff_color_hessian! methods for in-place and out-of-place computations, and a ForwardColorHesCache type containing buffers for computation. Feedback welcome!

To do:

• [x] Make sure there are method signatures corresponding to all the ones for Jacobians
• [x] More tests
• [x] Make sure performance is ok

[codecov-commenter]

Codecov Report

Merging #190 (e1c8eaf) into master (74d9fc6) will increase coverage by 1.20%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #190      +/-   ##
==========================================
+ Coverage   79.94%   81.14%   +1.20%     
==========================================
  Files          14       15       +1     
  Lines         753      801      +48     
==========================================
+ Hits          602      650      +48     
  Misses        151      151              

Impacted Files	Coverage Δ
src/SparseDiffTools.jl	100.00% <ø> (ø)
src/differentiation/compute_hessian_ad.jl	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 74d9fc6...e1c8eaf. Read the comment docs.

[ElOceanografo]

forwarddiff_color_jacobian! has default arguments for colorvec and sparsity, which I currently don't supply for forwarddiff_color_hessian!. Currently not sure what makes the most sense: 1) defaulting to an assumption that the Hessian is dense (i.e., colorvec = eachindex(x), sparsity = ones(length(x), length(x))), 2) doing automatic sparsity detection when they aren't specified, or 3) leaving it as-is and making the user explicitly supply the coloring and sparsity pattern.

[ChrisRackauckas]

1

[ElOceanografo]

I currently have two signatures for forwarddiff_color_hessian!:

• forwarddiff_color_hessian!(H, f, x, colorvec, sparsity) (creates a ForwardColorHesCache)
• forwarddiff_color_hessian!(H, f, x, hes_cache) (totally non-allocating)

There are also two corresponding signatures for forward_color_hessian, which create the sparse Hessian H instead of modifying it in place. I think this matches the interface for forwarddiff_color_jacobian!. Are any others needed?

[ElOceanografo]

Ok, I think this is getting pretty close. @ChrisRackauckas when you've got a second, lemme know what else you think it needs and I'll try to take care of it!

[ChrisRackauckas]

Is this still WIP? Looks close to done.