search

stan-dev / math

The Stan Math Library is a C++ template library for automatic differentiation of any order using forward, reverse, and mixed modes. It includes a range of built-in functions for probabilistic modeling, linear algebra, and equation solving.
https://mc-stan.org
BSD 3-Clause "New" or "Revised" License
752 stars 188 forks source link

ODE Methods not `fvar<T>` compatible #2839

Open andrjohns opened 2 years ago

andrjohns commented 2 years ago

Description

The ODE functors aren't compatible with fvar<T> inputs and fail to compile

Current Version:

v4.4.0

wds15 commented 2 years ago

That's no surprise. They have never been designed to be able to handle that as higher order derivatives are not in scope for them. It's not impossible to do, but getting first order to work is already a big challenge...higher order isn't a lot better...

WardBrian commented 2 years ago

Do any of the implicit functions support fvars? I was under the impression that none of them did, and that this was semi-intentional

charlesm93 commented 2 years ago

Implicit functions require bespoke methods to compute derivatives and it is often efficient to directly compute the vector-Jacobian product required for reverse-mode autodiff. It is also possible to write methods which directly compute Jacobian-vector products required for forward autodiff but there was no use-case in Stan. Beyond that, I suspect we can also derive methods which "directly" compute higher-order derivatives, e.g. by solving the right differential equation.

Technically, we have methods which compute the full Jacobian for ODEs (and once upon a time for algebraic solvers -- but this has been replaced with an adjoint method). The full Jacobian can then be contracted with a vector for either forward or reverse mode.

For ODEs, I think the RK45 and BDF methods require a linear solve, so a Jacobian is taken for the ODE RHS with respect to the input and this derivative is obtained via reverse mode ODE.

To corroborate @wds15 support for fvar is likely non-trivial, if technically feasible.

What is the motivation for using fvar in the ODE?

andrjohns commented 2 years ago

What is the motivation for using fvar in the ODE?

I've just been running the new hessian() method in cmdstanr across a bunch of example models to check for higher-order compatibility, and the ODE methods were the main failures (among many others).

It would be great if they were compatible, even if they weren't very performant - so that post-processing applications which need Hessians/second-order derivatives would work.

But I recognise that it's likely a bunch of work so there's no pressure from me!