ralna / RALFit

A non-linear least squares solver
Other
25 stars 6 forks source link

Add support for finite difference + derivative checker #61

Open talassio opened 5 years ago

talassio commented 5 years ago

Add support for derivative checker

The idea is to add and option (options%check_derivatives = .True.) that checks for:

  1. 1st order derivatives (user Jacobian vs FD approximation)
  2. 2nd order derivatives (if Hessian callback (eval_H) or Hessian times vector HP callback (eval_HP) is/are provided)
  3. If H and HP are provided then first check each of them against FD approximation
  4. Also add a feature to see/check if user is providing J^T instead of J and the flag options%fortran_Jacobian is not set correctly. Report back differences.

Add support for missing 1st order (possibly 2nd order) derivatives

Add support for completing missing entries from the user call-back of the Jacobian (eval_J) using finite differences, see Ceres chapter on derivatives.

This options should first check on the very first call which elements are provided and mark the missing ones to be approximated via FD, alternative it can check on every call (say only on first order callbacks) and approximate any required element.

talassio commented 4 years ago

References

C.J.F. Ridders, Accurate computation of F'(x) and F'(x) F"(x), Advances in Engineering Software (1978), Volume 4, Issue 2, April 1982, Pages 75-76, ISSN 0141-1195, http://dx.doi.org/10.1016/S0141-1195(82)80057-0.

Example

IPOpt: IpTNLPAdapter.cpp CheckDerivatives()

https://ceres-solver.googlesource.com/ceres-solver/+/refs/heads/master/include/ceres/internal/numeric_diff.h

LM.M provides a sketch on estimating via FD de Jacobian: https://people.duke.edu/~hpgavin/ce281/lm.pdf

MINPACK: lmdif

talassio commented 3 years ago

Add support for a derivatives checker via finite differences to test, Jacobian and Hessian Also add a feature to see/check if user is providing J^T instead of J and the flag options%fortran_Jacobian is set or not.