Bounds on parameters in Nelder-Mead

[zxjroger]

Hi, thank you for the package. I am confused about how to put bounds on parameters using Nelder-Mead. Could you please let me know which is the correct approach? Thank you.

lower = [-1.0, -1.0]
upper = [1.0, 1.0]
initial_x = [0.0, 0.0]

# First Approach
res = Optim.optimize(f, lower, upper, initial_x, Fminbox(NelderMead()), Optim.Options(show_trace = true))

# Second Approach
res = Optim.optimize(f, lower, upper, initial_x, NelderMead(), Optim.Options(show_trace = true))

# Third Approach
res = Optim.optimize(f, initial_x, NelderMead(lower = lower, upper = upper), Optim.Options(show_trace = true))

[pkofod]

Did you try them out? Did any work? :) I'm not sure I understand the question, sorry.

[longemen3000]

https://discourse.julialang.org/t/optim-problems-with-fminbox-and-the-neldermead-algorithm/60978/4

[bojeryd91]

I am having the same problem. A minimum working example:

f(x) = (x[1] - 2.0)^2 + (x[2] - 1.0)^2 + 1.0
res = optimize(f, [0.0, 0.0], [3.0, 3.0], [2.9, 2.9], Fminbox(NelderMead()))
println(res.minimum)  # Prints 10.250625
println(res.minimizer) # Prints [4.375, 2.9]

The solution should be approx. [2.0, 1.0], and all less than 3.0. But optimize returns a value outside the bounded box and it is not even a local minimum.

[bc0n]

This is still an issue in v1.7.1, here's the same example with trace on. What jumps out to me is the negative 'distance from box'. I don't see a specific test for Fminbox to compare against.

f(x) = (x[1] - 2.0)^2 + (x[2] - 1.0)^2 + 1.0
res = optimize(f, [0.0, 0.0], [3.0, 3.0], [2.9, 2.9], Fminbox(NelderMead()), Optim.Options(store_trace=true, show_trace=true))
println(res.minimum)  # Prints 10.250625
println(res.minimizer) # Prints [4.375, 2.9]

Fminbox
-------
Initial mu = 0.000270667

Fminbox iteration 1
-------------------
Calling inner optimizer with mu = 0.000270667

(numbers below include barrier contribution)
Iter     Function value    √(Σ(yᵢ-ȳ)²)/n 
------   --------------    --------------
     0     5.420000e+00              NaN
 * time: 0.02844095230102539

Exiting inner optimizer with x = [4.375, 2.9]
Current distance to box: -1.375
Decreasing barrier term μ.

Fminbox iteration 2
-------------------
Calling inner optimizer with mu = 2.70667e-7

(numbers below include barrier contribution)
Iter     Function value    √(Σ(yᵢ-ȳ)²)/n 
------   --------------    --------------
     0     5.420000e+00              NaN
 * time: 1.4066696166992188e-5

Exiting inner optimizer with x = [4.375, 2.9]
Current distance to box: -1.375
Decreasing barrier term μ.

10.250625
[4.375, 2.9]

[itsdfish]

I also encountered this issue. Are there any plans to fix this problem?

[jonlym]

Hi, I receive the same output as @bc0n when I run their MWE. Is there a fix or workaround to this issue?

Environment

julia> Pkg.status()
Status `~/Documents/julia/Fminbox_MWE/Project.toml`
  [429524aa] Optim v1.7.8

julia> VERSION
v"1.10.0"

Program Output

Fminbox
-------
Initial mu = 0.000270667

Fminbox iteration 1
-------------------
Calling inner optimizer with mu = 0.000270667

(numbers below include barrier contribution)
Iter     Function value    √(Σ(yᵢ-ȳ)²)/n 
------   --------------    --------------
     0     5.420000e+00              NaN
 * time: 0.026376962661743164

Exiting inner optimizer with x = [4.375, 2.9]
Current distance to box: -1.375
Decreasing barrier term μ.

Fminbox iteration 2
-------------------
Calling inner optimizer with mu = 2.70667e-7

(numbers below include barrier contribution)
Iter     Function value    √(Σ(yᵢ-ȳ)²)/n 
------   --------------    --------------
     0     5.420000e+00              NaN
 * time: 2.002716064453125e-5

Exiting inner optimizer with x = [4.375, 2.9]
Current distance to box: -1.375
Decreasing barrier term μ.

10.250625
[4.375, 2.9]