Speed and innacuracy of sciml_train with BFGS() on previously demonstrated code

[DiFiSi]

I'm really committed to use Julia for my future projects, but my learning curve has stalled for 3 days now because of this.

I was trying to implement a piece of code showcased live at Juliacon20 which used DiffEqFlux.sciml_train() with BFGS() to find the right parameters for the Lotka Volterra and the line runs without errors for hours without any kind of prints being shown in the REPL (I use VSCode but execution results are both shown inline and in the REPL). This happens even when I restrict the number of iterations. This is the code:

using DifferentialEquations, Plots

function lotka_volterra!(du,u,p,t)
    rab, wol = u
    α,β,γ,δ=p
    du[1] = drab = α*rab - β*rab*wol
    du[2] = dwol = γ*rab*wol - δ*wol
    nothing
end

u0 = [1.0,1.0]
tspan = (0.0,10.0)
p = [1.5,1.0,3.0,1.0]
prob = ODEProblem(lotka_volterra!,u0,tspan,p)
sol = solve(prob,saveat=0.1)
plot(sol)

dataset = Array(sol)
scatter!(sol.t,dataset')

tmp_prob = remake(prob, p=[1.2,0.8,2.5,0.8])
tmp_sol = solve(tmp_prob)
plot(tmp_sol)
scatter!(sol.t,dataset')

function loss(p)
    tmp_prob = remake(prob, p=p)
    tmp_sol = solve(tmp_prob,saveat=0.1)
    sum(abs2,Array(tmp_sol) - dataset)
end

using DiffEqFlux, Optim

pinit = [1.2,0.8,2.5,0.8]
res = DiffEqFlux.sciml_train(loss,pinit,BFGS())

Other tutorials run well both with Adam() and using BFGS(), such as Parameter Estimation on Highly Stiff Systems, but the results aren't exactly reproduced. Instead of getting this:

Ground truth: [0.04, 3.0e7, 10000.0]
Final parameters: [0.040002, 3.0507e7, 10084.0]
Error: 1.69%

I'm getting:

Ground truth: [0.04, 3.0e7, 10000.0]
Final parameters: [0.039997, 7.8891e9, 162080.0]
Error: 26200.0%

In the meantime, I've completely reinstalled everything Julia and came back to the same pieces of code, getting the same results. I've been looking for a while and I can't find anybody else with these issues related to sciml_train. Can anybody give me some pointers on what I might have gotten wrong?

[Vaibhavdixit02]

@DiFiSi for the first example if you change your loss to

function loss(p)
  tmp_prob = remake(prob, p=p)
  tmp_sol = Array(solve(tmp_prob,Tsit5(),saveat=0.1))
  if size(tmp_sol) == size(dataset)
    return sum(abs2,tmp_sol - dataset)
  else
    return Inf
  end
end

that should fix it for you, I am surprised it didn't error for you instead of running on, anyway, try this out and see if it fixes it for you.

For the other issue I can't recreate it

Ground truth: [0.04, 3.0e7, 10000.0]
Final parameters: [0.040017, 2.9784e7, 9968.3]
Error: 0.722%

can you confirm the versions of packages you are using? You can paste the ]st in this issue if you have a doubt

[DiFiSi]

Hey, @Vaibhavdixit02. I've tried your suggestion and I was able to reproduce the output of the code from JuliaCon20 - thanks! But, since the code I posted before was the exact one showcased and ran at the event by @ChrisRackauckas, why do you reckon I needed to implement your adaptation? Versioning variations?

Like you said, a dimension mismatch error does happen, but it is corrected when we specifiy the same saveat=N on both the solve for the dataset and the solve inside the loss, no?

As for the project state, it is

[2b5f629d] DiffEqBase v6.60.0
[aae7a2af] DiffEqFlux v1.36.1
[41bf760c] DiffEqSensitivity v6.44.1
[0c46a032] DifferentialEquations v6.16.0
[ced4e74d] DistributionsAD v0.6.24
[587475ba] Flux v0.12.2
[f6369f11] ForwardDiff v0.10.18
[429524aa] Optim v1.3.0
[1dea7af3] OrdinaryDiffEq v5.52.4
[91a5bcdd] Plots v1.12.0
[37e2e3b7] ReverseDiff v1.8.0

, while my Julia version is 1.6.1..

[Vaibhavdixit02]

I posted before was the exact one showcased and ran at the event by @ChrisRackauckas, why do you reckon I needed to implement your adaptation

I am not sure actually, the issue there was the intermediate parameters while running the optimisation were causing the ODE solve to fail, you must be seeing some warning for that. In that case the output of solve is not of the same dimension even if you specify saveat. It could be either because of changes in the optimisation or the differential equation solver used but it would be quite hard to pin it down.

[yha]

I've narrowed this down to the following:

using DifferentialEquations

function lotka_volterra!(du, (r,w), (α,β,γ,δ), t)
    du[1] = dr = α*r - β*r*w
    du[2] = dw = γ*r*w - δ*w
end

prob = ODEProblem(lotka_volterra!, [1.0, 1.0], (0.0, 10.0), [100, 700, -200, 1000])
solve(prob)

This solve seems to be stuck in an infinite loop. When run inside VSCode it can't even be interrupted, but in a standalone REPL it can:

julia> solve(prob)
ERROR: InterruptException:
Stacktrace:
  [1] derivative!(df::Vector{Float64}, f::SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, x::Float64, fx::Vector{Float64}, integrator::OrdinaryDiffEq.ODEIntegrator{CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, true, Vector{Float64}, Nothing, Float64, Vector{Int64}, Float64, Float64, Float64, Vector{Vector{Float64}}, OrdinaryDiffEq.ODECompositeSolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.CompositeInterpolationData{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}}, DiffEqBase.DEStats}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryMinHeap{Float64}, DataStructures.BinaryMinHeap{Float64}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Vector{Float64}, Float64, Nothing, OrdinaryDiffEq.DefaultInit}, grad_config::FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()})
    @ OrdinaryDiffEq ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\derivative_wrappers.jl:10
  [2] calc_tderivative!(integrator::OrdinaryDiffEq.ODEIntegrator{CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, true, Vector{Float64}, Nothing, Float64, Vector{Int64}, Float64, Float64, Float64, Vector{Vector{Float64}}, OrdinaryDiffEq.ODECompositeSolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.CompositeInterpolationData{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}}, DiffEqBase.DEStats}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryMinHeap{Float64}, DataStructures.BinaryMinHeap{Float64}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Vector{Float64}, Float64, Nothing, OrdinaryDiffEq.DefaultInit}, cache::OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}, dtd1::Float64, repeat_step::Bool)
    @ OrdinaryDiffEq ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\derivative_utils.jl:13
  [3] calc_rosenbrock_differentiation!
    @ ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\derivative_utils.jl:522 [inlined]
  [4] perform_step!(integrator::OrdinaryDiffEq.ODEIntegrator{CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, true, Vector{Float64}, Nothing, Float64, Vector{Int64}, Float64, Float64, Float64, Vector{Vector{Float64}}, OrdinaryDiffEq.ODECompositeSolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.CompositeInterpolationData{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}}, DiffEqBase.DEStats}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryMinHeap{Float64}, DataStructures.BinaryMinHeap{Float64}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Vector{Float64}, Float64, Nothing, OrdinaryDiffEq.DefaultInit}, cache::OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}, repeat_step::Bool)
    @ OrdinaryDiffEq ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\perform_step\rosenbrock_perform_step.jl:40
  [5] perform_step!
    @ ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\perform_step\composite_perform_step.jl:52 [inlined]
  [6] perform_step!
    @ ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\perform_step\composite_perform_step.jl:49 [inlined]
  [7] solve!(integrator::OrdinaryDiffEq.ODEIntegrator{CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, true, Vector{Float64}, Nothing, Float64, Vector{Int64}, Float64, Float64, Float64, Vector{Vector{Float64}}, OrdinaryDiffEq.ODECompositeSolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.CompositeInterpolationData{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}}, DiffEqBase.DEStats}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, OrdinaryDiffEq.CompositeCache{Tuple{OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, OrdinaryDiffEq.Rosenbrock23Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rosenbrock23Tableau{Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, Vector{Int64}}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, Vector{Int64}}, DefaultLinSolve, FiniteDiff.JacobianCache{Vector{Float64}, Vector{Float64}, Vector{Float64}, UnitRange{Int64}, Nothing, Val{:forward}(), Float64}, FiniteDiff.GradientCache{Nothing, Vector{Float64}, Vector{Float64}, Float64, Val{:forward}(), Float64, Val{true}()}}}, OrdinaryDiffEq.AutoSwitchCache{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryMinHeap{Float64}, DataStructures.BinaryMinHeap{Float64}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Vector{Float64}, Float64, Nothing, OrdinaryDiffEq.DefaultInit})
    @ OrdinaryDiffEq ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\solve.jl:455
  [8] __solve(::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, ::CompositeAlgorithm{Tuple{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}}, AutoSwitch{Tsit5, Rosenbrock23{0, false, DefaultLinSolve, DataType}, Rational{Int64}, Int64}}; kwargs::Base.Iterators.Pairs{Symbol, Bool, Tuple{Symbol, Symbol}, NamedTuple{(:default_set, :second_time), Tuple{Bool, Bool}}})
    @ OrdinaryDiffEq ~\.julia\packages\OrdinaryDiffEq\vxMSM\src\solve.jl:5
  [9] __solve(::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, ::Nothing; default_set::Bool, kwargs::Base.Iterators.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:second_time,), Tuple{Bool}}})
    @ DifferentialEquations ~\.julia\packages\DifferentialEquations\HSWeG\src\default_solve.jl:7
 [10] #__solve#71
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:282 [inlined]
 [11] __solve
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:269 [inlined]
 [12] #solve_call#56
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:61 [inlined]
 [13] solve_call
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:48 [inlined]
 [14] #solve_up#58
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:85 [inlined]
 [15] solve_up
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:75 [inlined]
 [16] #solve#57
    @ ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:70 [inlined]
 [17] solve(::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Int64}, ODEFunction{true, typeof(lotka_volterra!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem})
    @ DiffEqBase ~\.julia\packages\DiffEqBase\EeX3b\src\solve.jl:68
 [18] top-level scope
    @ REPL[4]:1

Apparently @Vaibhavdixit02 's version works because it specifies Tsit5().

[Vaibhavdixit02]

@ChrisRackauckas any idea what is happening here?

[ChrisRackauckas]

It's oscillating the stiffness detection so it's missing the instability detection: https://github.com/SciML/OrdinaryDiffEq.jl/pull/1358 .

[Vaibhavdixit02]

Doesn't seem like it since this is happening with current version of OrdinaryDiffEq as well. I am wondering if it has something to do with selection of the solver alg when it is not specified? Since using Tsit5 seems to work fine but the DifferentialEquations's solve without any alg seems to get stuck?

[ChrisRackauckas]

What? That would make it happen with the current version but not the previous, and if you print out integrator.dt you can see that this oscillation is stopping ending via dtmin. This has nothing to do with the selection of the algorithm and you can recreate it with just AutoTsit5(Rosenbrock23())

[gzagatti]

I can confirm that I am having the same problem. I was trying to follow JuliaCon 2020 tutorial and could not get it to work.

First, I ran into a number of issues --- --- somehow sciml_train was being blocked by the Tables package--- which forced me to basically remove the ~/.julia folder and start from scratch with a new Julia installation.

I then finally managed to replicate the optimization tutorial in DiffEqFlux page.

Once past these hurdles, I attempted to run the exact same code as @DiFiSi and got into an infinite loop as reported. The solution then proposed by @Vaibhavdixit02 worked and returned the correct parameters.

Echoing @DiFiSi, it would be useful to know why we were not able to replicate the tutorial.

[ChrisRackauckas]

Fixed in https://github.com/SciML/OrdinaryDiffEq.jl/pull/1398.

using DifferentialEquations, Plots

function lotka_volterra!(du,u,p,t)
    rab, wol = u
    α,β,γ,δ=p
    du[1] = drab = α*rab - β*rab*wol
    du[2] = dwol = γ*rab*wol - δ*wol
    nothing
end

u0 = [1.0,1.0]
tspan = (0.0,10.0)
p = [1.5,1.0,3.0,1.0]
prob = ODEProblem(lotka_volterra!,u0,tspan,p)
sol = solve(prob,saveat=0.1)
plot(sol)

dataset = Array(sol)
scatter!(sol.t,dataset')

tmp_prob = remake(prob, p=[1.2,0.8,2.5,0.8])
tmp_sol = solve(tmp_prob)
plot(tmp_sol)
scatter!(sol.t,dataset')

function loss(p)
  tmp_prob = remake(prob, p=p)
  tmp_sol = Array(solve(tmp_prob,Tsit5(),saveat=0.1))
  if size(tmp_sol) == size(dataset)
    return sum(abs2,tmp_sol - dataset)
  else
    return Inf
  end
end

using DiffEqFlux, Optim

pinit = [1.2,0.8,2.5,0.8]
res = DiffEqFlux.sciml_train(loss,pinit,BFGS())

@Vaibhavdixit02 what's the right tutorial to add the error check handling to?

[Vaibhavdixit02]

Not sure, I feel it might be a good idea to make the "Avoiding local minima" section to a general page of gotchas and tricks or something like that and add it there but not sure.

[ChrisRackauckas]

New tutorial: https://diffeqflux.sciml.ai/dev/examples/divergence/ and it's all fixed up. Thanks for the reports.