JuliaDiff / ReverseDiff.jl

Reverse Mode Automatic Differentiation for Julia
Other
348 stars 57 forks source link

Exploiting sparsity in higher-order differentiation computations #41

Open jrevels opened 7 years ago

jrevels commented 7 years ago

We should explore the use of techniques such as edge-pushing, graph coloring, etc. to discover and exploit sparsity patterns in ReverseDiff's second-order derivative computations.

Maybe sparsity optimization could be employed automagically whenever nested tapes are encountered?

mlubin commented 7 years ago

@fqiang and @cnpetra's experiments with edge pushing have been pretty successful at this point, benchmarks forthcoming. If you're looking to implement the state of the art, I'd vote for edge pushing over the graph coloring approaches.