Add the ability to store the trace for nonlinear solve algorithms

[avik-pal]

Fixes #58 (kind of). This is meant to add tracing capability to native NonlinearSolve solvers

Checklist

• [x] Any code changes were done in a way that does not break public API
• [x] All documentation related to code changes were updated
• [x] The new code follows the contributor guidelines, in particular the ScioML Style Guide and COLPRAC.
• [x] Upstream Patches
  • [x] DiffEqBase needs 3 new kwargs
  • [x] SciMLBase needs an additional storage for trace
  • [x] Bump compact entries
• [x] Extend to all solvers
  • [x] NR
  • [x] TR
  • [x] GN
  • [x] LM
  • [x] Broyden
  • [x] Klement
  • [x] PT
• [x] Display which algorithm is printing the trace (needed for polyalgorithms)
• [x] Add option to log every k iterations

Additional context

Add any other context about the problem here.

Example Usage

julia> solve(prob; show_trace = Val(true), trace_level = TraceAll(150))

Algorithm: GeneralKlement(linsolve = LUFactorization())

----     -------------        -----------          -------             
Iter     f(u) inf-norm        Step 2-norm          cond(J)             
----     -------------        -----------          -------             
0        3.50000000e+07       0.00000000e+00       1.00000000e+00      
Final    3.50000000e+07      
----------------------      

Algorithm: GeneralBroyden()

----     -------------        -----------          -------             
Iter     f(u) inf-norm        Step 2-norm          cond(J)             
----     -------------        -----------          -------             
0        3.50000000e+07       0.00000000e+00       1.00000000e+00      
Final    3.50000000e+07      
----------------------      

Algorithm: NewtonRaphson(ad = AutoForwardDiff())

----     -------------        -----------          -------             
Iter     f(u) inf-norm        Step 2-norm          cond(J)             
----     -------------        -----------          -------             
0        3.50000000e+07       0.00000000e+00       1.31206964e+13      
Final    3.50000000e+07      
----------------------      

Algorithm: NewtonRaphson(ad = AutoForwardDiff(), linesearch = BackTracking())

----     -------------        -----------          -------             
Iter     f(u) inf-norm        Step 2-norm          cond(J)             
----     -------------        -----------          -------             
0        3.50000000e+07       0.00000000e+00       3.12767074e+05      
150      1.96894976e+07       1.62771357e+01       2.54208181e+09      
300      1.95822332e+07       8.59874893e+00       3.54255023e+09      
450      1.95254850e+07       5.64893544e+00       5.30212288e+09      
600      1.94937971e+07       4.75390720e+00       7.69360853e+09      
750      1.94830888e+07       2.36799303e+00       1.23415632e+10      
900      1.94776278e+07       1.58184438e+00       1.84208351e+10      
Final    1.94758224e+07      
----------------------      

Algorithm: TrustRegion(ad = AutoForwardDiff(), radius_update_scheme = Simple)

----     -------------        -----------          -------             
Iter     f(u) inf-norm        Step 2-norm          cond(J)             
----     -------------        -----------          -------             
0        3.50000000e+07       0.00000000e+00       8.15768984e+11      
150      1.76071235e+07       1.55362216e+03       2.50639043e+12      
300      1.76071235e+07       1.55362216e+03       2.50639043e+12      
450      1.76071235e+07       1.55362216e+03       2.50639043e+12      
600      1.76071235e+07       1.55362216e+03       2.50639043e+12      
750      1.76071235e+07       1.55362216e+03       2.50639043e+12      
900      1.76071235e+07       1.55362216e+03       2.50639043e+12      
Final    1.76071235e+07      
----------------------      

Algorithm: TrustRegion(ad = AutoForwardDiff(), radius_update_scheme = Bastin)

----     -------------        -----------          -------             
Iter     f(u) inf-norm        Step 2-norm          cond(J)             
----     -------------        -----------          -------             
0        3.50000000e+07       0.00000000e+00       8.16936903e+11      
150      8.33325158e-01       2.82025866e-03       1.58940933e+11      
300      8.33325110e-01       2.82025866e-03       1.61054537e+11      
450      8.33325063e-01       2.82025866e-03       1.63225116e+11      
600      8.33325017e-01       2.82025866e-03       1.65455004e+11      
750      8.33324971e-01       2.82025866e-03       1.67746665e+11      
900      8.33324926e-01       2.82025866e-03       1.70102704e+11      
Final    8.33324896e-01      
----------------------      
u: 3-element Vector{Float64}:
   29.8340911480401
 2349.223979895471
 1271.6287520542758

[codecov[bot]]

Codecov Report

Attention: 30 lines in your changes are missing coverage. Please review.

Comparison is base (46912f2) 94.04% compared to head (1b9c188) 93.07%.

Files	Patch %	Lines
src/trace.jl	79.06%	18 Missing :warning:
src/trustRegion.jl	56.25%	7 Missing :warning:
src/NonlinearSolve.jl	90.47%	2 Missing :warning:
src/utils.jl	66.66%	2 Missing :warning:
src/gaussnewton.jl	75.00%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #292 +/- ## ========================================== - Coverage 94.04% 93.07% -0.97% ========================================== Files 20 21 +1 Lines 1896 2065 +169 ========================================== + Hits 1783 1922 +139 - Misses 113 143 +30 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.