Certain `Optim.jl` solvers such as `NelderMead()` return solutions that silently violate bounds. These still give `SciMLBase.retcode_successful` and can be caught here

[avinashresearch1]

Minimal Reproducible Example 👇

f(x, p = nothing) = - 0.17416892598381517 - 0.04003064633247909*x[1] + 0.04583730611180919*x[2] + 0.005229901524626298*x[1]^2 + 0.0017265452658399075*x[1]*x[2] + 0.0017180217574026728*x[2]^2 + 0.000161111153452011*x[1]^3 + 0.0006761257889020228*x[1]^2*x[2] + 0.0008832150720625505*x[1]*x[2]^2 + 0.0001704260769958016*x[2]^3
lb = [0.744094729423523, -1.0]
ub = [1.0, -0.755813717842102]
x0 = (lb .+ ub) ./ 2

opt_f = OptimizationFunction(f, Optimization.AutoForwardDiff())
opt_prob = OptimizationProblem(opt_f, x0; lb = lb, ub = ub, sense = MinSense)  
sol = solve(opt_prob, NelderMead())

>retcode: Success
u: 2-element Vector{Float64}:
  1.3330710470676421 # This violates bounds
 -0.877906858921051

julia> sol.u .< ub
2-element BitVector:
 0
 1

julia> lb .< sol.u
2-element BitVector:
 1
 1

One way to deal with this in the Optimization.jl rather than in Optim.jl could be to test for this violation before passing on the retcode successful to the user.

  [6a86dc24] FiniteDiff v2.23.1
  [26cc04aa] FiniteDifferences v0.12.32
  [f6369f11] ForwardDiff v0.10.36
  [b6b21f68] Ipopt v1.6.5
  [b4f0291d] LazySets v2.14.1
⌃ [961ee093] ModelingToolkit v9.28.0
⌃ [7f7a1694] Optimization v3.26.3
  [3e6eede4] OptimizationBBO v0.3.0
  [fd9f6733] OptimizationMOI v0.4.2
  [36348300] OptimizationOptimJL v0.3.2
  [1dea7af3] OrdinaryDiffEq v6.87.0
  [91a5bcdd] Plots v1.40.5
  [0bca4576] SciMLBase v2.48.1
  [e88e6eb3] Zygote v0.6.70
  [37e2e46d] LinearAlgebra

[Vaibhavdixit02]

yeah that makes sense, more generally the automatic wrapping in Fminbox has not been very effective with nelder mead methods. Does this become better with some hyperparameter fiddling?

[Vaibhavdixit02]

Do you want to open a PR for this?