Porting over some low cost Algorithms

[avik-pal]

• [x] Broyden
  • [x] Inplace
  • [x] OOP
  • [x] Integrate into the Test Suite
  • [x] Basic Tests
  • [x] 23 Test Problems
• [x] Klement
  • [x] Inplace
  • [x] OOP
  • [x] Integrate into the Test Suite
  • [x] Basic Tests
  • [x] 23 Test Problems

[codecov[bot]]

Codecov Report

Merging #257 (bd5af95) into master (0dac21a) will decrease coverage by 1.66%. The diff coverage is 91.04%.

@@            Coverage Diff             @@
##           master     #257      +/-   ##
==========================================
- Coverage   93.90%   92.25%   -1.66%     
==========================================
  Files          15       18       +3     
  Lines        1231     1614     +383     
==========================================
+ Hits         1156     1489     +333     
- Misses         75      125      +50     

Files	Coverage Δ
src/NonlinearSolve.jl	90.00% <100.00%> (+0.52%)	:arrow_up:
src/default.jl	77.21% <ø> (-1.27%)	:arrow_down:
src/dfsane.jl	97.90% <ø> (-2.10%)	:arrow_down:
src/levenberg.jl	98.46% <100.00%> (ø)
src/pseudotransient.jl	100.00% <100.00%> (ø)
src/raphson.jl	100.00% <100.00%> (ø)
src/trustRegion.jl	95.85% <100.00%> (-3.51%)	:arrow_down:
src/lbroyden.jl	98.37% <98.37%> (ø)
src/broyden.jl	96.05% <96.05%> (ø)
src/utils.jl	84.32% <89.65%> (+1.15%)	:arrow_up:
... and 2 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[avik-pal]

For Klement if we allow any general linear solve, how do we go about doing the singularity check for resetting? For LU, we could do the check on the diagonal of U, but that won't work for other solvers. Should we use cond(J) > inv(tol), or is there a better way to do this?

cc @ChrisRackauckas

[avik-pal]

I have added both Klement and Broyden. The naming is a bit unfortunate since SimpleNonlinearSolve reserves the names instead of Simple****.

Broyden is faster than the one in SimpleNonlinearSolve.

Klement as expected seems to spend a significant amount of time in singularity checking. Someone with more familiarity with LinearAlgebra needs to chime in on how to handle this in the general case. I can add a branching based on LU factorization and default to that.

[avik-pal]

I am reusing the factorization for singularity if the user specifies LUFactorization, else we fallback to ArrayInterface.issingular

julia> @benchmark solve(probN, GeneralKlement(; linsolve=LUFactorization()))
BenchmarkTools.Trial: 23 samples with 1 evaluation.
 Range (min … max):  168.920 ms … 283.473 ms  ┊ GC (min … max): 0.00% … 40.56%
 Time  (median):     218.867 ms               ┊ GC (median):    1.57%
 Time  (mean ± σ):   219.950 ms ±  37.673 ms  ┊ GC (mean ± σ):  8.37% ± 11.92%

  ███           ▁ █   ▁   ▁ ▁     ▁   █ ▁ ▁      ▁█ ▁     ▁   ▁  
  ███▁▁▁▁▁▁▁▁▁▁▁█▁█▁▁▁█▁▁▁█▁█▁▁▁▁▁█▁▁▁█▁█▁█▁▁▁▁▁▁██▁█▁▁▁▁▁█▁▁▁█ ▁
  169 ms           Histogram: frequency by time          283 ms <

 Memory estimate: 252.29 MiB, allocs estimate: 201.

julia> @benchmark solve(probN, GeneralKlement(; linsolve=QRFactorization()))
BenchmarkTools.Trial: 12 samples with 1 evaluation.
 Range (min … max):  382.058 ms … 475.142 ms  ┊ GC (min … max): 1.69% … 20.59%
 Time  (median):     440.708 ms               ┊ GC (median):    1.52%
 Time  (mean ± σ):   433.578 ms ±  36.193 ms  ┊ GC (mean ± σ):  5.63% ±  7.69%

  █ ▁           ▁                    ▁ ▁ ▁       ▁ ▁        ▁ █  
  █▁█▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁█▁█▁▁▁▁▁▁▁█▁█▁▁▁▁▁▁▁▁█▁█ ▁
  382 ms           Histogram: frequency by time          475 ms <

 Memory estimate: 309.65 MiB, allocs estimate: 229.

julia> @benchmark solve(probN, GeneralKlement(; linsolve=nothing))
BenchmarkTools.Trial: 18 samples with 1 evaluation.
 Range (min … max):  232.665 ms … 382.726 ms  ┊ GC (min … max):  0.87% … 36.40%
 Time  (median):     274.173 ms               ┊ GC (median):     2.18%
 Time  (mean ± σ):   283.057 ms ±  47.992 ms  ┊ GC (mean ± σ):  10.11% ± 12.59%

  █▁   ▁                                                         
  ██▁▁▁█▁▁▁▁▁▁▆▁▁▁▁▁▁▁▁▆▁▁▁▁▁▆▁▁▁▁▁▁▆▆▁▆▆▆▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ ▁
  233 ms           Histogram: frequency by time          383 ms <

 Memory estimate: 305.70 MiB, allocs estimate: 192.

on a problem with 1000 states.

[avik-pal]

Marking it as WIP, I want to add LBroyden to this as well!

[avik-pal]

@ChrisRackauckas this is ready to go!

[avik-pal]

@ChrisRackauckas let's merge this one?