search

SciML / OrdinaryDiffEq.jl

High performance ordinary differential equation (ODE) and differential-algebraic equation (DAE) solvers, including neural ordinary differential equations (neural ODEs) and scientific machine learning (SciML)
https://diffeq.sciml.ai/latest/
Other
521 stars 198 forks source link

Optimistic first time convergence criterion #2183

Closed YingboMa closed 3 weeks ago

YingboMa commented 2 months ago

CVODE estimates the convergence of the first iteration by reusing the convergence rate of the previous nonlinear solver iteration.

Reference: https://github.com/LLNL/sundials/blob/2abd63bd6cbc354fb4861bba8e98d0b95d65e24a/src/cvodes/cvodes_nls.c#L325-L331

MWE: https://discourse.julialang.org/t/cvode-bdf-outperforms-julia-solvers-for-stiff-system-biology-model/113936

Master branch:

julia> sol = solve(oprob_jac, FBDF()).stats
SciMLBase.DEStats
Number of function 1 evaluations:                  311
Number of function 2 evaluations:                  0
Number of W matrix evaluations:                    31
Number of linear solves:                           222
Number of Jacobians created:                       2
Number of nonlinear solver iterations:             212
Number of nonlinear solver convergence failures:   0
Number of fixed-point solver iterations:                     0
Number of fixed-point solver convergence failures:           0
Number of rootfind condition calls:                0
Number of accepted steps:                          84
Number of rejected steps:                          2

This PR:

julia> sol = solve(oprob_jac, FBDF()).stats
SciMLBase.DEStats
Number of function 1 evaluations:                  251
Number of function 2 evaluations:                  0
Number of W matrix evaluations:                    32
Number of linear solves:                           159
Number of Jacobians created:                       2
Number of nonlinear solver iterations:             149
Number of nonlinear solver convergence failures:   0
Number of fixed-point solver iterations:                     0
Number of fixed-point solver convergence failures:           0
Number of rootfind condition calls:                0
Number of accepted steps:                          86
Number of rejected steps:                          3
ChrisRackauckas commented 2 months ago

CVODE estimates the convergence of the first iteration by reusing the convergence rate of the previous nonlinear solver iteration.

wild! I wonder how that effects SDIRK.

YingboMa commented 2 months ago

Yeah, this should make SDIRK methods more competitive against BDF methods and Rosenbrock methods as well.

YingboMa commented 2 months ago

θ Here's the evolution of the convergence rate for this particular problem. I guess constant extrapolation isn't a bad guess to capture the plateaus.

YingboMa commented 2 months ago

θ-t Here's theta vs model time.

YingboMa commented 2 months ago

The test failures seem unrelated.

devmotion commented 2 months ago

AFAICT the DelayDiffEq test failures are caused by this PR.

ChrisRackauckas commented 2 months ago

type FPSolver has no field prev_θ

Yes I'm very sure that's related to this PR.

ChrisRackauckas commented 2 months ago

For reference, only Interface I is failing on master, and I hope to get to that this weekend.

YingboMa commented 2 months ago

Many of the regression tests might be too stringent.

ChrisRackauckas commented 1 month ago

https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/9004907852/job/24739004289?pr=2183#step:6:799

Some of the convergence tests are showing big red flags. I don't think it's just a tolerance thing.

ChrisRackauckas commented 1 month ago

Is more logic for step rejections required? I.e. can you use the convergence rate of a rejected step?

oscardssmith commented 1 month ago

The DelayDiffEq failures seem real (although the tests that are failing do seem pretty bad)