Add switch for global vs local solvers (status codes)

[rschwarz]

As discussed in #17, we could add a switch to specify that the given solver is able to find a global optimum and decide infeasibility globally (e.g. BARON, KNITRO, SCIP) or will only give results that are valid locally (e.g. IPOPT).

[rschwarz]

Currently, the implicit assumption (given in the default values for the keyword arguments termination_target and primal_target) is that the solver only has local capabilities.

That is, the termination codes that appear in the current tests are:

• LOCALLY_SOLVED for feasible problems,
• LOCALLY_INFEASIBLE for infeasible problems.

For a global solver, I would expect OPTIMAL and INFEASIBLE (or maybe INFEASIBLE_OR_UNBOUNDED), respectively.

But for the primal result codes, it's a little bit more complicated. Currently, the tests use:

• FEASIBLE_POINT for feasible problems,
• INFEASIBLE_POINT for infeasible problems.

FEASIBLE_POINT is also a good status for global solvers, but INFEASIBLE_POINT might as well be NO_SOLUTION. But that latter difference is independent of the local/global capablities of the solver, and more a difference of solving method (iterative vs spatial branching?).

So, maybe we need more than one dimension of characterizing the solver?

The signature of the test functions themselves could be changed in a way that there is one keyword argument to signal whether we expect the problem to be feasible vs infeasible, rather than giving the codes for termination and primal result explictly.

[rschwarz]

I also wonder whether we should make a distinction between convex and non-convex problems. That is, shouldn't a local solver find the globally optimal solution for a convex problem, and be able to return OPTIMAL, rather than just LOCALLY_SOLVED? Of course, that depends on whether the solver knows about the problem's convexity...

[rschwarz]

A quick sketch before I start a proper PR:

# We only distinguish between feasible and infeasible problems now.
@enum ProblemTypeCode FEASIBLE_PROBLEM INFEASIBLE_PROBLEM

# Optimizers can decide either local or global optimality.
@enum SolverCapabilityCode LOCAL GLOBAL

"""
    OptimizerCapability

A value of the enum OptimizerCapabilityCode.
"""
struct OptimizerCapability <: JuMP.MOI.AbstractOptimizerAttribute end

# Optimizers have (assumed) local capabilities, unless specified explicitly.
JuMP.MOI.get(optimizer::AbstractOptimizer, ::OptimizerCapability) = LOCAL

TERMINATION_TARGET = Dict(
    (FEASIBLE_PROBLEM, GLOBAL) => JuMP.MOI.OPTIMAL,
    (FEASIBLE_PROBLEM, LOCAL) => JuMP.MOI.LOCALLY_SOLVED,
    (INFEASIBLE_PROBLEM, GLOBAL) => JuMP.MOI.INFEASIBLE,
    (INFEASIBLE_PROBLEM, LOCAL) => JuMP.MOI.LOCALLY_INFEASIBLE,
)

PRIMAL_TARGET = Dict(
    (FEASIBLE_PROBLEM, GLOBAL) => JuMP.MOI.FEASIBLE_POINT,
    (FEASIBLE_PROBLEM, LOCAL) => JuMP.MOI.FEASIBLE_POINT,
    # not really a property of local/global?!
    (INFEASIBLE_PROBLEM, GLOBAL) => JuMP.MOI.NO_SOLUTION,
    (INFEASIBLE_PROBLEM, LOCAL) => JuMP.MOI.INFEASIBLE_POINT,
)

That way, we would need to change all test case functions, removing the current keywork arguments for the status codes, but adding a ::ProblemTypeCode.

It also means that any given optimizer (type) can only have either LOCAL or GLOBAL capability, so we can not expect different behavior per test-case.

So, it might be better to just pass down the value for the capabiliy, together with the solver tolerances.

[odow]

Note that the tests already support over-riding the statuses: https://github.com/JuliaOpt/MINLPTests.jl/blob/364c105c0153ea766de2af93b76f28b044c8e8b1/src/nlp-cvx/001_010.jl#L1-L3

It's just a matter of hooking it up here: https://github.com/JuliaOpt/MINLPTests.jl/blob/364c105c0153ea766de2af93b76f28b044c8e8b1/src/MINLPTests.jl#L72-L82 and https://github.com/JuliaOpt/MINLPTests.jl/blob/364c105c0153ea766de2af93b76f28b044c8e8b1/src/MINLPTests.jl#L103-L125

[rschwarz]

I don't quite follow your suggestions.

Note that the tests already support over-riding the statuses:

Sure, but I can't simply override the statuses of all test cases (using the same values). For example, in some cases, the default value termination_status would be replaced with MOI.OPTIMAL, in other cases with MOI.INFEASIBLE.

So, the correct target status is a function of both the optimizer and the problem in the test case.

[odow]

Sure, but I can't simply override the statuses of all test cases (using the same values). For example, in some cases, the default value termination_status would be replaced with MOI.OPTIMAL, in other cases with MOI.INFEASIBLE.

Good point. So maybe a LOCAL/GLOBAL switch is best. In solver wrappers can always exclude certain tests from the run and test them individually with a more specific status if needed.

[rschwarz]

In solver wrappers can always exclude certain tests from the run and test them individually with a more specific status if needed.

Hm, yeah. I don't see a way of making the change I propose without also changing the signature of all of the test functions (to include a FEASIBLE/INFEASIBLE specification). If we also want to keep the possibility to override the target statuses of individual tests, the logic of defaults and exceptions becomes too hairy, I think.