SLSQP in NLopt.jl 1.0.2 throws error: `bug: more than iter SQP iterations`

[thorek1]

I understand 1.0.1 => 1.0.2 brought changes to the error handling.

this is inconvenient in the case of SLSQP because it (now) throws an error when maxeval is reached and the optimisation failed. for my purposes it would be useful to have access to the return values even if the optimisation failed/errored. in other words, not throw an error and hand over return values

[stevengj]

That's weird, it's supposed to never reach that check, since we check the evaluation count elsewhere.

[thorek1]

I linked the wrong line. now corrected to L2585

I don't fully grasp the flow of the script (yet) but I can confirm that he throws: bug: more than iter SQP iterations

[odow]

Do you have a reproducible example?

[mzaffalon]

I get the same error. Here is a MWE:

using NLopt

ξ = [-1.0, -0.8019385609518437, -0.4450484640237961, 0.0, 0.4450484640237961, 0.8019385609518437, 1.0]
weight = [1.309924793995232e7, 1.311697371095455e7, 1.31290303653434e7, 1.3134417169089135e7, 7.300760397632944e6, 296.9188842891679, 1.0]
invy = [7.634026049312568e-8, 7.623709721739069e-8, 7.61670871475545e-8, 7.613584882574196e-8, 1.369720338068101e-7, 0.003367923203652162, 1.0]
p = [0.0, 0.021220788737679505, 0.4893496181966227, -0.27747890271555137]

opt = Opt(:LD_SLSQP, 4)
opt.lower_bounds = [-Inf, 0, 0, -Inf]
opt.ftol_rel = 1e-3

function myfunc(p::Vector, grad::Vector)
    p[2] < 0 && (p[2] = 0)
    p[3] < 0 && (p[3] = 0)
    Δ = sqrt.(p[2] .+ p[3]*(ξ.-p[4]).^2)
    f = p[1] .+ Δ
    objf = (invy .- f) .* weight
    if length(grad) > 0 # do not use isempty(grad)
        grad[1] =       -2sum(objf                      .* weight)
        grad[2] =        -sum(objf                 ./ Δ .* weight)
        grad[3] =        -sum(objf .* (ξ.-p[4]).^2 ./ Δ .* weight)
        grad[4] = 2p[3] * sum(objf .* (ξ.-p[4])    ./ Δ .* weight)
    end
    sum(abs2, objf)
end

opt.min_objective = myfunc
(minf, minx, ret) = optimize(opt, p)

NLopt v1.0.2 and Julia 1.10.3.

[danielwe]

Experiencing the same issue, but only if I add inequality constraints, suggesting that this may be a fallthrough from case 4 which is reported as a case 9. Is there a problem with the sign convention for constraints here? The original Fortran documentation indicates that the SLSQP code expects c(x) >= 0, while NLopt's convention is the opposite: c(x) <= 0.

https://github.com/stevengj/nlopt/blob/7a7587e5ef1cb15f412515d852d1fc261c863e96/src/algs/slsqp/slsqp.c#L2222

[odow]

I can confirm the failure, but this is an issue that should probably be reported to the upstream: https://github.com/stevengj/nlopt

It has been reported before: https://github.com/stevengj/nlopt/issues/215, but it is helpful to have the MWE.

[odow]

Here's a smaller example:

julia> using NLopt

julia> function my_objective_fn(p::Vector, grad::Vector)
           if length(grad) > 0
               grad .= [1e8 * p[2]^2, 2e8 * p[1] * p[2]]
           end
           return 1e8 * (p[1] * p[2]^2)
       end
my_objective_fn (generic function with 1 method)

julia> opt = Opt(:LD_SLSQP, 2)
Opt(LD_SLSQP, 2)

julia> lower_bounds!(opt, [0, -Inf])

julia> min_objective!(opt, my_objective_fn)

julia> optimize(opt, [1, -1])
ERROR: nlopt failure FAILURE: bug: more than iter SQP iterations
Stacktrace:
 [1] error(::String, ::String)
   @ Base ./error.jl:44
 [2] chk(o::Opt, result::Result)
   @ NLopt ~/.julia/packages/NLopt/w0c7n/src/NLopt.jl:227
 [3] optimize!(o::Opt, x::Vector{Float64})
   @ NLopt ~/.julia/packages/NLopt/w0c7n/src/NLopt.jl:630
 [4] optimize(o::Opt, x::Vector{Int64})
   @ NLopt ~/.julia/packages/NLopt/w0c7n/src/NLopt.jl:634
 [5] top-level scope
   @ REPL[524]:1

[odow]

And here it is using ccalls:

julia> using NLopt

julia> NLopt.NLOPT_VERSION
v"2.8.0"

julia> function my_scalar_callback_fn(n, p_x, p_grad, ::Ptr{Cvoid})::Cdouble
           x = unsafe_wrap(Array, p_x, (n,))
           if p_grad !== C_NULL
               grad = unsafe_wrap(Array, p_grad, (n,))
               grad .= [1e8 * x[2]^2, 2e8 * x[1] * x[2]]
           end
           return 1e8 * (x[1] * x[2]^2)
       end
my_scalar_callback_fn (generic function with 1 method)

julia> c_my_scalar_callback_fn = @cfunction(
           my_scalar_callback_fn,
           Cdouble,
           (Cuint, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cvoid})
       )
Ptr{Nothing} @0x0000000105e16300

julia> opt = NLopt.nlopt_create(NLopt.NLOPT_LD_SLSQP, 2)
Ptr{Nothing} @0x00006000009ae800

julia> NLopt.nlopt_set_lower_bounds(opt, [0, -Inf])
NLOPT_SUCCESS::nlopt_result = 1

julia> NLopt.nlopt_set_min_objective(opt, c_my_scalar_callback_fn, C_NULL)
NLOPT_SUCCESS::nlopt_result = 1

julia> opf_f = Ref{Cdouble}(NaN)
Base.RefValue{Float64}(NaN)

julia> NLopt.nlopt_optimize(opt, [1.0, -1.0], opf_f)
NLOPT_FAILURE::nlopt_result = -1

julia> unsafe_string(NLopt.nlopt_get_errmsg(opt))
"bug: more than iter SQP iterations"

[odow]

I have a C reproducer in https://github.com/stevengj/nlopt/issues/215#issuecomment-2303303204

[odow]

Closing in favor of https://github.com/stevengj/nlopt/issues/215