FastDifferentiation (FD) is a package for generating efficient executables to evaluate derivatives of Julia functions. It can also generate efficient true symbolic derivatives for symbolic analysis.
Unlike forward and reverse mode automatic differentiation FD automatically generates efficient derivatives for arbitrary function types: ℝ¹->ℝ¹, ℝ¹->ℝᵐ, ℝⁿ->ℝ¹, and ℝⁿ->ℝᵐ, m≠1,n≠1. FD is similar to D* in that it uses the derivative graph[^a] but FD is asymptotically faster so it can be applied to much larger expression graphs.
For f:ℝⁿ->ℝᵐ with n,m large FD may have better performance than conventional AD algorithms because the FD algorithm finds expressions shared between partials and computes them only once. In some cases FD derivatives can be as efficient as manually coded derivatives (see the Lagrangian dynamics example in the D* paper or the Benchmarks section of the documentation for another example).
FD may take much less time to compute symbolic derivatives than Symbolics.jl even in the ℝ¹->ℝ¹ case. The executables generated by FD may also be much faster (see the documentation for more details).
You should consider using FastDifferentiation when you need:
This is the FD feature set:
Dense Jacobian | Sparse Jacobian | Dense Hessian | Sparse Hessian | Higher order derivatives | Jᵀv | Jv | Hv | |
Compiled function | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Symbolic expression | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Jᵀv and Jv compute the Jacobian transpose times a vector and the Jacobian times a vector, without explicitly forming the Jacobian matrix. For applications see this paper. Hv computes the Hessian times a vector without explicitly forming the Hessian matrix.
See the documentation for more information on the capabilities and limitations of FD.
If you use FD in your work please share the functions you differentiate with me. I'll add them to the benchmarks. The more functions available to test the easier it is for others to determine if FD will help with their problem.
Q: Does FD support complex numbers?
A: Not currently.
Q: You say FD computes efficient derivatives but the printed version of my symbolic derivatives is very long. How can that be efficient?
A: FD stores and evaluates the common subexpressions in your function just once. But, the print function recursively descends through all expressions in the directed acyclic graph representing your function, including nodes that have already been visited. The printout can be exponentially larger than the internal FD representation.
Q: How about matrix and tensor expressions?
A: If you multiply a matrix of FD variables times a vector of FD variables the matrix vector multiplication loop is effectively unrolled into scalar expressions. Matrix operations on large matrices will generate large executables and long preprocessing time. FD functions with up 10⁵ operations should still have reasonable preprocessing/compilation times (approximately 1 minute on a modern laptop) and good run time performance.
Q: Does FD support conditionals?
A: Yes, but see the documentation for limitations. Full functionality will come in a future release.
[^a]: See the D* paper for an explanation of derivative graph factorization.