search

SciML / DifferentialEquations.jl

Multi-language suite for high-performance solvers of differential equations and scientific machine learning (SciML) components. Ordinary differential equations (ODEs), stochastic differential equations (SDEs), delay differential equations (DDEs), differential-algebraic equations (DAEs), and more in Julia.
https://docs.sciml.ai/DiffEqDocs/stable/
Other
2.86k stars 228 forks source link

Questions about using automatic differentiation to get the local sensitivity by an adaptive ODE solver #749

Closed yilinyang1 closed 3 years ago

yilinyang1 commented 3 years ago

Hi, I have a question about using the automatic differentiation to get the local sensitivity by an adaptive ODE solver. For some ODE solvers, the stepsize might depend on the parameters of the ODE system. So, when taking the derivative of the ODE solution to the parameters, there will be an extra term dh/dp (h is the stepsize and p is the parameter), which is not desired. How does this package deal with this issue? Thanks!

ChrisRackauckas commented 3 years ago

The solution of an ODE is not dependent on the stepsize of the approximation, so for continuous adjoints that term is not in there. I recommend going through the derivation at https://github.com/mitmath/18337 for details. Let me know if that helps.

yilinyang1 commented 3 years ago

Actually, my question comes from this paper. When we use automatic differentiation instead of the continuous adjoints sensitivity analysis, it is likely that the derivative to the parameters through the step size are included (like equation 2.2 in the paper).

ChrisRackauckas commented 3 years ago

By default it modifies the AD to enforce that the dh_i = 0. It actually has to turn that back on if ContinuousCallbacks are used though, in which case you have to differentiate the callback time, and this is done only in the callback mechanism to make it consistent. This was all done back in https://github.com/SciML/DiffEqBase.jl/pull/156

The paper is still actually incorrect because you do still need to have different time steps in the differentiation form in order to force gradients to be correct. https://github.com/SciML/DiffEqSensitivity.jl/issues/273 is such an issue where the method in the paper would get essentially infinite error, which is fixed by https://github.com/SciML/DiffEqBase.jl/pull/529 .

yilinyang1 commented 3 years ago

I see. It's very helpful. Thanks a lot!