Changes to the NLSolverseBase

[jlperla]

• [x] Get rid of the "Numerical gradients" section, will want to show the OnceDifferentiable way instead.
• [x] Remove the x from the structure and f
• [x] Need to make the OnceDifferentiable stored value generic. See https://docs.julialang.org/en/v1/manual/types/#Parametric-Types-1

  struct NLoptAdapter <: Function
  f
  x
  nlsolver_base::OnceDifferentiable
  end

  Something like

  struct NLoptAdapter{T} <: Function where {T <: AbstractObjective}
  f
  x
  nlsolver_base::T
  end

• [x] Does this have anything to do with NLopt? or is it just changing the direction.
  • Adds two things: swapping the order and making it callable.
• [x] Get it working, then try to call a unconstrained maximization with a derivative-free algorithm. change opt(:LD_LBFGS algorithm.
• [x] Additional: solve the same problem, stacking the two constraints to make NLopt have a vector-valued constraint, and add as a test. See how the AD works with the stacking. May not call fdf?
• [x] Add the :central to tests for the central differences.

[jlperla]

Is there a better way to do

# compute gradient by forward automatic differentiation
function g!(G::Vector, x::Vector)
    ForwardDiff.gradient!(G, f, x)
end

function fg!(x::Vector, grad::Vector)
    if length(grad) > 0 # gradient of f(x)
        g!(grad, x)
    end
    f(x)
end

Solving for the function and the gradient at the same time?

[chiyahn]

Resolved at cbe604b1811f3a0e8981baff4c7c1149d4c18b06.