Incumbent solution not returned when time limit hit for GUROBI

[apschmidt]

Details for the issue

What did you do?

Ran pulp using the GUROBI solver and set a time limit.

What did you expect to see?

After time limit hit, the best incumbent solution accessible using value() function.

What did you see instead?

All variables are None types.

Useful extra information

Appears to be related to the following issue: https://github.com/coin-or/pulp/issues/445

The info below often helps, please fill it out if you're able to. :)

What operating system are you using?

• [X] Windows: ( version: ___ )
• [ ] Linux: ( distro: ___ )
• [ ] Mac OS: ( version: ___ )
• [ ] Other: ___

I'm using python version:

• [ ] 2.7
• [ ] 3.4
• [ ] 3.5
• [ ] 3.6
• [X] Other: _3.9__

I installed PuLP via:

• [ ] pypi (python -m pip install pulp)
• [X] github (python -m pip install -U git+https://github.com/coin-or/pulp)
• [ ] Other: ___ (conda?)

Did you also

• [X] Tried out the latest github version: https://github.com/coin-or/pulp
• [X] Searched for an existing similar issue: https://github.com/coin-or/pulp/issues?utf8=%E2%9C%93&q=is%3Aissue%20

[pchtsp]

Did you check the status of the solution / solver? What was it? It may be that the solver did not find any solution in the time limit.

[apschmidt]

Here is the last few lines of the output. A feasible solution was indeed found by the solver.

[pchtsp]

Mmm interesting. Do you have a reproducible example? For example, a json / mps export of the model already solved / without solving ? Also, the complete gurobi log in a file?

[beevabeeva]

Hi, I just ran into a similar issue. It's also similar to #586 .

In my case, variables could be retrieved as usual after a solve for a small problem size:

    solver = getSolver('GUROBI', msg=1,  timeLimit=time_limit)
    model.solve(solver) 

    for v in model.variables():
        if v.varValue != 0:
          print(v.name,': ',v.varValue)

However, when solving a large problem, all variables had value None after solving (and the solver did run, found 10 solutions etc.).

Workaround

I was reading through an old version of the source code, and saw this:

            #populate pulp solution values
            for var in lp.variables():
                try:
                    var.varValue = var.solverVar.X

If I change my variable printing code above (in the first code block) to use solverVar.X instead of varValue as follows:

  for v in model.variables():
    if v.solverVar.X != 0:
      print(v.name,': ',v.solverVar.X)

then I can successfully access the variable values!

So the issue does seem to be linked to the time limit in some way (need to do a little extra testing to confirm) but when the problem is small, the solver finishes before the time limit (TBC), which would hint as to why it works for the small problem but not the large.

I am using: pulp version 2.7 gurobipy version 10.0.0 Python version 3.10.9 OS is ubuntu (in a docker container)

Hope this workaround helps others and more importantly, leads to a proper solution.

Note:

This problem does NOT occur for CPLEX or CBC (for any problem size)

[apschmidt]

@beevabeeva Do you know if the time limit is being hit for the small instance?

Your workaround seems to access the Gurobi solver's variables directly rather than the Pulp's variable values. It looks like this is because Pulp model is only updating its variable values if an optimal solution is found (LpStatusOptimal) while the status GRB.TIME_LIMIT maps to LpStatusNotSolved.

[apschmidt]

Mmm interesting. Do you have a reproducible example? For example, a json / mps export of the model already solved / without solving ? Also, the complete gurobi log in a file?

My example is not shareable, but maybe @beevabeeva's is?

[flippercy]

@apschmidt , @pchtsp:

I have a related question on how timelimit works for pulp. Assume I set timelimit = 600 seconds while the solver find the optimal solution in 300 seconds, what will the solver do? Stop solving and return the solution after 300 seconds or keep searching and return the solution after 600 seconds?

Thank you!

[apschmidt]

@flippercy I believe this depends on how the specific solver is designed, since pulp will pass the time limit to the solver you are using. Every solver that I have ever used would return a solution after 300 seconds (i.e., the time limit only matters if an optimal solution is NOT found before that time limit is reached; if the optimal is found before the time limit, it would just return the solution right away).

[flippercy]

@flippercy I believe this depends on how the specific solver is designed, since pulp will pass the time limit to the solver you are using. Every solver that I have ever used would return a solution after 300 seconds (i.e., the time limit only matters if an optimal solution is NOT found before that time limit is reached; if the optimal is found before the time limit, it would just return the solution right away).

Thank you @apschmidt ! And if no global optimal solution is found within the time limit, the best current feasible solution will be returned as the result, is it correct?

[apschmidt]

@flippercy I believe this depends on how the specific solver is designed, since pulp will pass the time limit to the solver you are using. Every solver that I have ever used would return a solution after 300 seconds (i.e., the time limit only matters if an optimal solution is NOT found before that time limit is reached; if the optimal is found before the time limit, it would just return the solution right away).

Thank you @apschmidt ! And if no global optimal solution is found within the time limit, the best current feasible solution will be returned as the result, is it correct?

Correct, that is the typical behavior.

[flippercy]

@apschmidt @pchtsp I am asking this question because sometimes Pulp returns an empty result while the problem is feasible. Not sure whether it is due to the time limit or other bugs.

[apschmidt]

@apschmidt @pchtsp I am asking this question because sometimes Pulp returns an empty result while the problem is feasible. Not sure whether it is due to the time limit or other bugs.

This question is related to the top level comment on this thread - it appears there may be some solvers (Gurobi is an example) for which the underlying solver has found a feasible (not proven optimal) solution by the time limit, but Pulp does not properly propagate that solution to the user. So it could be either a bug or that no feasible solution has been found by the time limit. You can typically look at the output log to determine if a feasible solution has been found.

[pchtsp]

It's probably a bug. It should be a simple fix of the status mapping after the solving. If anyone has a gurobi license and finds the time to do a PR with this small change, I'd appreciate it!

[ronaldvdv]

Indeed the same #586 ; I identified the source of the problem but am in doubt whether the current design of PuLP would be sufficient, when only differentiating between "solved" (=optimal, solution retrieved) and "not solved" (=time limit, no solution retrieved). Should there be a third state (feasible solution but optimality not proven)?

[apschmidt]

That would be ideal but would require a large amount of refactoring.

[ronaldvdv]

Do you foresee any problems if we take the shortcut and define "TIME_LIMIT" in Gurobi as "solved" in PuLP (potentially depending on an additional check if there is actually an incumbent)?