Adaptive Preconditioner

[mjohnson541]

Particularly for large simulations where the system is large enough you can't construct the jacobian explicitly and need to rely on methods like GMRES time spent in the solver (rather than in functional evaluations) can dominate. In this case it helps a lot to have a good preconditioner. Fortunately, this has already been studied here for chemical kinetic systems: https://doi.org/10.1016/j.proci.2014.05.113 the preconditioning interface for CVODE_BDF is described here: https://diffeq.sciml.ai/latest/solvers/ode_solve/#ode_solve_sundials-1.

[jiweiqi]

Hi @ChrisRackauckas , is there something already existing in DifferentialEquations.jl that does a similar thing for acceleration. Like keyword options that allow us to do it.

[ChrisRackauckas]

https://diffeq.sciml.ai/stable/tutorials/advanced_ode_example/#Declaring-a-Jacobian-Free-Newton-Krylov-Implementation

https://diffeq.sciml.ai/stable/features/linear_nonlinear/#iterativesolvers-jl

Shows how to define linear solvers and add preconditioners. Indeed it would be good to get this preconditioner in a SciML repo and make it one of the default switches in the linear solver interface.

[jiweiqi]

@ChrisRackauckas Do you expect such kind of preconditioners can also accelerate the adjoint sensitivity, either in adjoint or forwarddiff. To me, my major pain now is the slow gradient computation comparing while I feel the forward pass is already pretty fast.

[ChrisRackauckas]

No. For the most part a direct method should be faster until you get to like >1000 ODEs. I don't know many reaction systems that get there. https://www.sciencedirect.com/science/article/abs/pii/S1540748914001163?via%3Dihub isn't a great example because it isn't comparing apples to apples.

The average computation times for integrating the constant volume WSR are compared for three approaches in Figure 1. The first approach is based on the traditional ODE solvers used in chemical kinetics, which can still be found in some multi-dimensional CFD codes (e.g., Kiva3V-MZ [5]). The second approach uses the commercial package >CHEMKIN-PRO, which offers an advanced ODE solver developed by Reaction Design that takes advantage of the sparsity of detailed kinetic mechanisms. This particular solver is available in Reaction Design’s FORTE´ CFD code, while other CFD codes like CONVERGE and some of the newest Kiva variants (building on the work of Perini [9]) also take similar advantage of sparse solvers. The >third approach is based on the adaptive preconditioner developed for this investigation with the improved exponential functions and solver settings for CVODE and SuperLU from [27]

It will win out sooner or later, but what you really want to see is the improved everything with CVODE using sparse direct SuperLU vs the adaptive preconditioner form. That should have a cutoff around 1000 around or (from what we've seen with other preconditioners). The paper has the cutoff closer too 100, but against methods that we know are not efficient against sparse direct CVODE.

But back to the sensitivity, this means when n+p > 1000.

To me, my major pain now is the slow gradient computation comparing while I feel the forward pass is already pretty fast.

How are you doing the gradient computation? The details there are fairly important. We can probably discuss this off of this issue though.

[mjohnson541]

Resolved with #185