RFC: Many solvers are/will be the same

[KnutAM]

and only differ by the matrix they require. Right now we have two solvers, but they have exactly the same fields:

struct NewtonSolver{LS,LSearch,T}
    linsolver::LS
    linesearch::LSearch
    maxiter::Int 
    tolerance::T
    numiter::Vector{Int}  # Last step number of iterations
    residuals::Vector{T}  # Last step residual history
end
Base.@kwdef struct SteepestDescent{LineSearch,LinearSolver,T}
    linsolver::LinearSolver = BackslashSolver()
    linesearch::LineSearch = ArmijoGoldstein()
    maxiter::Int = 200
    tolerance::T = 1e-6
    numiter::Vector{Int} = [zero(maxiter)]  # Last step number of iterations
    residuals::Vector{T} = zeros(typeof(tolerance),maxiter+1)  # Last step residual history
end

They only differ in default values and the matrix required from getsystemmatrix. For all methods that have this pattern, perhaps it would be possible/suitable to just have a matrixtype field that could have a singelton type (e.g. TrueJacobian, DescentPreconditioner, etc. ), for example

struct NewtonLike{MT,LSolve,LSearch,T}
    matrixtype::MT
    linsolver::LSolve
    linesearch::LSearch
    maxiter::Int 
    tolerance::T
    numiter::Vector{Int}  # Last step number of iterations
    residuals::Vector{T}  # Last step residual history
end

Then, getsystemmatrix(problem, matrixtype) could be used instead as well, which is in my mind nicer than giving the type. From my perspective, then we could get rid of the special names getjacobian, getdescentpreconditioner and just use this, perhaps with the following defaults getsystemmatrix(problem,::Any)=getsystemmatrix(problem) getsystemmatrix(::Any,::DescentPreconditioner) = I (Noting that if a user defines getsystemmatrix(::MyProblem,::Any) it will give ambiguity, but I think that's fine as they should then rather define getsystemmatrix(::MyProblem) or getsystemmatrix(::MyProblem, ::TheMatrixType))