Open tle4336 opened 1 year ago
Dr. Forrest here seems to indicate, from his own run, that the problem is with pulp
, not with cbc
. However, looking at the disrepancy displayed from the part of cbc
, I am thinking otherwise. Anyone could give some insights into this weird issue of getting feasible
versus infeasible
when running cbc on Windows 10 from Pycharm v.s. Linux? https://github.com/coin-or/Cbc/issues/608
@dims @dpo @wookenny @chmduquesne How can we turn off the presolve
option of CBC solver via pulp
? I tried following this link but the presolve step still occurs, which led to infeasibility of my LP.
https://or.stackexchange.com/questions/8285/how-do-i-turn-off-preprocessing-in-pulp
I had a quick look at the code of the library and it looks like presolve is set to "on" as soon as the value passed is different from None. My guess is that you are explicitly passing "False" and it achieves the opposite effect from what you want.
See https://github.com/coin-or/pulp/blob/master/pulp/apis/coin_api.py#L230
I would recommend setting it to None and see what happens.
To debug this, you may want to check what the COIN_CMD object returns when you invoke its method getOptions().
If this fixes the problem, please report a bug as this is counter intuitive behavior and would need to be rewritten.
Cheers, Christophe-Marie
On Tue, Jul 11, 2023, 22:42 tle4336 @.***> wrote:
@dims https://github.com/dims @dpo https://github.com/dpo @wookenny https://github.com/wookenny @chmduquesne https://github.com/chmduquesne How can we turn off the presolve option of CBC solver via pulp? I tried following this link but the presolve step still occurs, which led to infeasibility of my LP.
https://or.stackexchange.com/questions/8285/how-do-i-turn-off-preprocessing-in-pulp
— Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/668#issuecomment-1631482178, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABZIMHCEBOVV5GUFYL6GATXPW3DDANCNFSM6AAAAAA2GFT6UU . You are receiving this because you were mentioned.Message ID: @.***>
@chmduquesne Thank you so much. I will try out setting the presolve
to None
(you are spot-on that I set it to False
).
However, I am not sure what you meant with "check what the COIN_CMD object returns"? The only solver object I successfully initiated was PULP_CBC_CMD
(e.g. solver = pulp.PULP_CBC_CMD(presolve=None). So you meant I should check solver.getOptions()
?
@chmduquesne Just a quick update. I tried to set presolve=None
within solver=pulp.PULP_CBC_CMD(presolve=None)
,and then call it via optimize.solve(solver)
. Still see the presolve step being conducted (and again, the result from presolve was wrong, as compared to the result of Dr. Forrest).
@tle4336 did you check what solver.getOptions() returns?
Just to add that setting the presolve
parameter to None
will at best not set the presolve option. If the command line tool has it on
by default, that will not work. I would recommend trying to patch getOptions to look like this:
def getOptions(self):
params_eq = dict(
gapRel="ratio {}",
gapAbs="allow {}",
threads="threads {}",
presolve="presolve {}",
strong="strong {}",
cuts="gomory on knapsack on probing on",
timeMode="timeMode {}",
)
options = self.optionsDict.copy()
if "presolve" in options:
if options["presolve"] in (True, "on"):
options["presolve"] = "on"
else:
options["presolve"] = "off"
return [
v.format(options[k])
for k, v in params_eq.items()
if self.optionsDict.get(k) is not None
]
I have too little time to test and submit a proper PR, but feel free to experiment on your end and try to make your way to the contributors list :smiley:
@chmduquesne Thank you so much for your sharing. I tried it out with your great instructions, but the added code above unfortunately did not stop the presolve
from being executed. Currently, this makes CBC solver judged multiple feasible
problems as "infeasible" (attached are two .mps
files of those problems).
2InfeasibleProbs.zip
@chmduquesne I also checked solver.getOptions()
but it did not give out any additional info besides the main message coming out of CBC solver.
solver.getOptions()
should return a formatted list of options to be passed to the solver, which should contain "presolve off"
Is the solver an object such as pulp.PULB_COIN_CMD()? Because of that is the case, my solver.getOptions() did not show such list of options that you are talking about?
On Thu, Jul 13, 2023 at 03:30 Christophe-Marie Duquesne < @.***> wrote:
solver.getOptions() should return a formatted list of options to be passed to the solver, which should contain "presolve off"
— Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/668#issuecomment-1633715954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHDDNX2U4AGO5V33K3GQJSLXP6P2PANCNFSM6AAAAAA2GFT6UU . You are receiving this because you were mentioned.Message ID: @.***>
I decided to see if I could reproduce your bug, and I can not. This is how I import the mps you provided:
import pulp
_, prob = pulp.LpProblem.fromMPS("/tmp/python_1.mps")
solver = pulp.COIN_CMD(presolve=None)
prob.solve(solver)
print("Status:", pulp.LpStatus[prob.status])
No infeasibility reported. This prints the following:
Welcome to the CBC MILP Solver
Version: 2.10.7
Build Date: Feb 14 2022
command line - cbc /tmp/d5cc7b65cea54ec986db0321eca69eb3-pulp.mps timeMode elapsed branch printingOptions all solution /tmp/d5cc7b65cea54ec986db0321eca69eb3-pulp.sol (default strategy 1)
At line 2 NAME MODEL
At line 3 ROWS
At line 378 COLUMNS
At line 2851 RHS
At line 3225 BOUNDS
At line 3638 ENDATA
Problem MODEL has 373 rows, 412 columns and 2060 elements
Coin0008I MODEL read with 0 errors
Option for timeMode changed from cpu to elapsed
Presolve 206 (-167) rows, 313 (-99) columns and 1077 (-983) elements
0 Obj 28881.81 Primal inf 166.23454 (117)
77 Obj 59379.1
Optimal - objective value 59379.1
After Postsolve, objective 59379.1, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 59379.0999 - 77 iterations time 0.002, Presolve 0.00
Option for printingOptions changed from normal to all
Total time (CPU seconds): 0.01 (Wallclock seconds): 0.01
Status: Optimal
As far as I can tell:
cbc /tmp/d5cc7b65cea54ec986db0321eca69eb3-pulp.mps timeMode elapsed branch printingOptions all solution /tmp/d5cc7b65cea54ec986db0321eca69eb3-pulp.sol
does not contain any mention of presolve, yet a presolve runs by default, as demonstrated by the line Presolve 206 (-167) rows, 313 (-99) columns and 1077 (-983) elements
presolve=False
to the solver). Here is the result of the execution if I do so
Welcome to the CBC MILP Solver
Version: 2.10.7
Build Date: Feb 14 2022
command line - cbc /tmp/540f706ca3374050bd44fbc8c5b57362-pulp.mps presolve off timeMode elapsed branch printingOptions all solution /tmp/540f706ca3374050bd44fbc8c5b57362-pulp.sol (default strategy 1) At line 2 NAME MODEL At line 3 ROWS At line 378 COLUMNS At line 2851 RHS At line 3225 BOUNDS At line 3638 ENDATA Problem MODEL has 373 rows, 412 columns and 2060 elements Coin0008I MODEL read with 0 errors Option for presolve changed from on to off Option for timeMode changed from cpu to elapsed 0 Obj 0 Primal inf 255.52382 (186) 82 Obj 58822.83 Primal inf 11.471553 (39) 131 Obj 59379.1 Optimal - objective value 59379.1 Optimal objective 59379.0999 - 131 iterations time 0.002 Option for printingOptions changed from normal to all Total time (CPU seconds): 0.00 (Wallclock seconds): 0.00
Status: Optimal
(note the line `Option for presolve changed from on to off`)
* You provided the wrong mps file. I also checked running cbc directly and get similar results.
@chmduquesne Thank you very much for rigorously testing the initial problem provided!! I highly appreciate it.
To your last point: the provided mps file was correct in my case. When running CBC solver on my machine (Windows 10 64-bit with pycharm's 2021 version) by either explicitly calling solver = pulp.PULP_CBC_CMD()' and
optimize.solve(solver)in which
optimizeis an object containing the LP, or just go with
optimize.solve(), the result given back from CBC solver was
infeasible. But when running that same problem with
lpsolvepackage in R, the result is
feasible.Is
pulp.COIN_CMD()and
pulp.PULP_CBC_CMD()` the same? Because if that is not, this could be the reason.
I saw that in your output, the version of CBC MILP solver is 2.10.7
(mine is 2.10.3
). So it looks like 2.10.7
works better then. Can you help show me how we can upgrade this solver in Python?
Finally, Could you please help try these 3 .mps
files in the .zip
file below, to see if you also got a feasible optimal result? Mine was still infeasible
in ALL 3 cases.
InfeasibleProblems.zip
@chmduquesne So it is not PULP_CBC_CMD()
, but rather COIN_CMD()
to invoke the code change in turning off the presolve
?
The class PULP_CBC_CMD
derives from COIN_CMD
. The only difference is the binary they invoke: PULP_CBC_CMD
invokes a precompiled version of cbc
that ships with pulp, and COIN_CMD
invokes a cbc
independently installed on your machine. In my case, I have installed cbc
separately via my linux package manager.
If I pass an instance of PULP_CBC_CMD
as the solver (instead of COIN_CMD
), I get the following results:
With presolve on [Edit: I had pasted the wrong output]
Welcome to the CBC MILP Solver
Version: 2.10.3
Build Date: Dec 15 2019
command line - /home/chmd/code/pulp/venv/lib/python3.10/site-packages/pulp/solverdir/cbc/linux/64/cbc /tmp/7a347b95ee7440d1af79d4497ca79759-pulp.mps timeMode elapsed branch printingOptions all solution /tmp/7a347b95ee7440d1af79d4497ca79759-pulp.sol (default strategy 1)
At line 2 NAME MODEL
At line 3 ROWS
At line 378 COLUMNS
At line 2851 RHS
At line 3225 BOUNDS
At line 3638 ENDATA
Problem MODEL has 373 rows, 412 columns and 2060 elements
Coin0008I MODEL read with 0 errors
Option for timeMode changed from cpu to elapsed
Presolve 206 (-167) rows, 313 (-99) columns and 1077 (-983) elements
0 Obj 28881.81 Primal inf 166.23454 (117)
77 Obj 59379.1
Optimal - objective value 59379.1
After Postsolve, objective 59379.1, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 59379.0999 - 77 iterations time 0.002, Presolve 0.00
Option for printingOptions changed from normal to all
Total time (CPU seconds): 0.00 (Wallclock seconds): 0.00
Status: Optimal
With presolve off (forced via my patch)
Welcome to the CBC MILP Solver
Version: 2.10.3
Build Date: Dec 15 2019
command line - /home/chmd/code/pulp/venv/lib/python3.10/site-packages/pulp/solverdir/cbc/linux/64/cbc /tmp/d648f8256d6042f09b2c81b486ed75d6-pulp.mps presolve off timeMode elapsed branch printingOptions all solution /tmp/d648f8256d6042f09b2c81b486ed75d6-pulp.sol (default strategy 1)
At line 2 NAME MODEL
At line 3 ROWS
At line 378 COLUMNS
At line 2851 RHS
At line 3225 BOUNDS
At line 3638 ENDATA
Problem MODEL has 373 rows, 412 columns and 2060 elements
Coin0008I MODEL read with 0 errors
Option for presolve changed from on to off
Option for timeMode changed from cpu to elapsed
0 Obj 0 Primal inf 255.52382 (186)
82 Obj 58822.83 Primal inf 11.471553 (39)
131 Obj 59379.1
Optimal - objective value 59379.1
Optimal objective 59379.0999 - 131 iterations time 0.002
Option for printingOptions changed from normal to all
Total time (CPU seconds): 0.00 (Wallclock seconds): 0.00
Status: Optimal
As you can see, via PULP_CBC_CMD
, I am using the same version as yours: 2.10.3
.
If I were you, I would verify that the cbc
binary that ships with your local copy of pulp really thinks that you mps file is infeasible. As you can see, the command line that pulps run is in the debug output, so you can edit it to run against the file you provided.
Details for the issue
What did you do?
I am currently encountering a very weird issue with the
PULP_CBC_CMD
solver viapulp
package in Python (CBC MILP Solver - version 2.10.3). Despite playing around with the parameter options gapAbs and fracGap, I had no way to make the CBC MILP solver to avoid running into infeasible status. However, the optimization model is actually feasible (the model is an LP problem, with all variables being continuous between 0 and 1), as it was solved quickly and easily via lpsolve package in R (with optimal value =112268.4
). I then compared the two models and could confirm that they are identical (so, the part of building the model in 2 languages is fine). I am wondering if anyone could help me figure out how to fix this incorrect issue?Attached is the model in .lp and .mps format, exported from both Python and R (you can see the filename to determine which one is exported from Python). Downloads.zip
I also include the .txt log file given by the CBC run. I also compute the values of the constraints as well as their corresponding slacks below:
What did you expect to see?
Expect to see the
LpStatus[optimize.status]
to returnOptimal
with optimal value close to112268.4
.What did you see instead?
Problem is given as
Infeasible
!See the log - log.txt
Useful extra information
The info below often helps, please fill it out if you're able to. :)
What operating system are you using?
I'm using python version:
I installed PuLP via:
Did you also