Warm Start feature not working for CBC with PuLP on Windows

[sachin-4099]

Even though warmStart=True is set it does not display/print these lines: mipstart values read for 3213 variables. Cbc0045I mipstart provided solution with cost 12

If these lines are not getting printed, is this an indication that the provided feasible solution is not getting loaded?

I am using the same python code as provided here in the documentation on how to warm start a solver.

Note: Works fine on Linux but not on Windows (tested on 64-bit).

This is the output generated on running the above python script:

Welcome to the CBC MILP Solver
Version: 2.10
Build Date: Jan  3 2021

At line 2 NAME          MODEL
At line 3 ROWS
At line 23 COLUMNS
At line 24708 RHS
At line 24727 BOUNDS
At line 27941 ENDATA
Problem MODEL has 18 rows, 3213 columns and 15062 elements
Coin0008I MODEL read with 0 errors
opening mipstart file .\Local\Temp\3d92e16fc503432c97984231532ed3fa-pulp.mst.
Continuous objective value is 12 - 0.02 seconds
Cgl0004I processed model has 18 rows, 3213 columns (3213 integer (3213 of which binary)) and 15062 elements
Cutoff increment increased from 1e-05 to 0.9999
Cbc0038I Initial state - 0 integers unsatisfied sum - 0
Cbc0038I Solution found of 12
Cbc0038I Before mini branch and bound, 3213 integers at bound fixed and 0 continuous
Cbc0038I Mini branch and bound did not improve solution (0.17 seconds)
Cbc0038I After 0.19 seconds - Feasibility pump exiting with objective of 12 - took 0.05 seconds
Cbc0012I Integer solution of 12 found by feasibility pump after 0 iterations and 0 nodes (0.20 seconds)
Cbc0001I Search completed - best objective 12, took 0 iterations and 0 nodes (0.20 seconds)
Cbc0035I Maximum depth 0, 0 variables fixed on reduced cost
Cuts at root node changed objective from 12 to 12
Probing was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Gomory was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Knapsack was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Clique was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
MixedIntegerRounding2 was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
FlowCover was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
TwoMirCuts was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
ZeroHalf was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)

Result - Optimal solution found

Objective value:                12.00000000
Enumerated nodes:               0
Total iterations:               0
Time (CPU seconds):             0.34
Time (Wallclock seconds):       0.34

Option for printingOptions changed from normal to all
Total time (CPU seconds):       0.53   (Wallclock seconds):       0.53

The choosen tables are out of a total of 3213:
('M', 'N')
('E', 'F', 'G')
('A', 'B', 'C', 'D')
('I', 'J', 'K', 'L')
('O', 'P', 'Q', 'R')

[OmarAbdo]

Hi Sachin, Did you find any solution to this problem? Does going back to older versions solve it?

[pchtsp]

mmm it does read the mip start file so I think pulp is doing what it should do:

opening mipstart file .\Local\Temp\3d92e16fc503432c97984231532ed3fa-pulp.mst.

The example in the docs is not the best example because it provides an optimal solution to a small problem. It may well be that CBC finds and proves the optimal solution before even loading the mipstart file. Have you tested this on larger problems? If there's indeed something wrong maybe check the CBC github for an answer: https://github.com/coin-or/Cbc/

@tkralphs any thoughts on why linux and windows show slightly different messages for the mipstart?

[sachin-4099]

Another issue I want to point out is: Even when I am providing an optimal solution with the help of warmstart, I am not able to get the answer instantaneously, using CBC solver with PuLP.

[tkralphs]

I'm not sure what's going on here, I would need to dig into it a bit. I will note, though, that providing the optimal solution may not always help a lot and is certainly not expected to reduce the computation to an instantaneous one. Cbc has no way of knowing that the solution is actually optimal just by examining it, so it still needs to do a search to determine whether there exists an improving solution somewhere in the feasible region. Even when there are no improving solutions, it may not be that easy to prove this. Much of the effort of the algorithm goes into constructing the proof that the solution is optimal.

[pchtsp]

I'm going to close the issue because everything seems to work from pulp's side. Please create a new issue if you don't agree.

[MLopez-Ibanez]

The issue is a bug in CBC: https://github.com/coin-or/Cbc/issues/32

Running with keepFiles=True uses a relative path and it does not trigger the bug.

Not sure if Pulp can workaround the issue by calling CBC with a relative path in Windows, but it could emit a warning that warmStart=True and keepFiles=False does not work for CBC in Windows.

[MLopez-Ibanez]

Could this be reopened so that people can find the work-around? Or should I open a new issue?

[pchtsp]

of course! Also, feel free to propose a warning on the warmStart=True / keepFiles=False config. Does this problem depend on the version of CBC?

[rwoodama]

I have what I believe to be the same issue. We are solving a very large LP and are hoping that warm starting will help, but at least providing a solution much closer to the optimal. The attachment is a toy version that generates its own data. I run it in both warm start on and off and contrasting also with keeping variables continuous or integer. CBC says it opens the mip start file but then proceeds with identical log sequence of actions as when it is not asked to warm start. I posted this exact same issue on the CBC site and John says that saving a ,lp file directly from cbc and reopening worked. PulP doesn't appear to be able to do that.
The PuLP route I have tried to implement is write a csv with variable values at the end of a run to be then be read back in on the next loop through and use that to set variable values. Mechanically, that sees to be working, but after reading in the MIP file, CBC doesn't use the solution, but proceeds as if nothing happened. Thoughts?

WarmStartTestCode.txt

[MLopez-Ibanez]

I have what I believe to be the same issue.

If the issue is not fixed by using "keepFiles=True", then it is not the same issue.

[rwoodama]

El Buen Hombre MLopez-Ibanez has pointed out the obvious. So tried it. Behold, keepFiles=True allows cbc to not just import the mipstart file but to then read the solution. This was very encouraging, and it worked for a smaller problem which is a modified version of the wedding table sample code where it will not provide integer solutions from the LP relaxation (attached). That problem takes 15 seconds to solve on the first pass (no warm start, no keepFiles) and about 5 seconds second pass. Success! TweakedWeddingSeatingProblemWarmStart.txt

But in my bigger (but still small compared to the BIG problem), alas it complained. Here's just a portion of the log: command line - C:\Users\fwoodaman\Documents\IRONMCtests\CPMRMT_Tool_20210716\code\IRON_MC_Py37_ENV\lib\site-packages\pulp\apis..\solverdir\cbc\win\64\cbc.exe IRON_WS_Test-pulp.mps max mips IRON_WS_Test-pulp.mst branch printingOptions all solution IRON_WS_Test-pulp.sol (default strategy 1) At line 2 NAME MODEL At line 3 ROWS At line 282 COLUMNS At line 3207 RHS At line 3485 BOUNDS At line 4094 ENDATA Problem MODEL has 277 rows, 608 columns and 1568 elements Coin0008I MODEL read with 0 errors opening mipstart file .\IRON_WS_Test-pulp.mst. MIPStart values read for 608 variables. Continuous objective value is 475.158 - 0.01 seconds Cgl0003I 1 fixed, 192 tightened bounds, 0 strengthened rows, 0 substitutions Cgl0003I 0 fixed, 64 tightened bounds, 0 strengthened rows, 0 substitutions Cgl0004I processed model has 106 rows, 356 columns (356 integer (40 of which binary)) and 921 elements Cbc0045I Fixing only non-zero variables. Cbc0045I Warning: mipstart values could not be used to build a solution. Cbc0012I Integer solution of -460.67171 found by DiveCoefficient after 751 iterations and 0 nodes (3.49 seconds)....

It proceeded to find the solution following the same sequence of nodes.

So I have discovered it works for some cases and that the fix for that is keepFiles=True. Why it decides to not work for mine remains to be send.

Thank you for pointing out the obvious Manuel!