Open cwfparsonson opened 3 years ago
For reference, here is the item_placement_1.mps
file contents:
Downloadable .mps:
https://drive.google.com/file/d/1rpOx4GomiPzSry733hIXtCW1yccmG1iD/view?usp=sharing
Simple .txt (can view in browser):
https://drive.google.com/file/d/1RKYo63FYPvAyFM-R8Au48TLQLC3v9zsx/view?usp=sharing
I just downloaded the MPS file, read it into Cbc without errors and solved the instance in a few seconds. So the problem is at least not with the MPS file. I'll leave it to others to determine whether the problem is with PuLP itself. Depending on what you're actually trying to do, there are other ways of reading an MPS file in Python, such as CyLP.
Hello everyone!
The problem probably is with pulp. The functionality is quite recent and although we have tested it with some mps files we probably have not done an exhaustive test.
I'll try to check what's wrong at the end of this week and get back to you.
On Wed, Jul 7, 2021, 04:27 Ted Ralphs @.***> wrote:
I just downloaded the MPS file, read it into Cbc without errors and solved the instance in a few seconds. So the problem is at least not with the MPS file. I'll leave it to others to determine whether the problem is with PuLP itself. Depending on what you're actually trying to do, there are other ways of reading an MPS file in Python, such as CyLP.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/459#issuecomment-875220759, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ454RC7UKO2N4TUW4UTTWO3PPANCNFSM475DGCWQ .
I just downloaded the MPS file, read it into Cbc without errors and solved the instance in a few seconds. So the problem is at least not with the MPS file. I'll leave it to others to determine whether the problem is with PuLP itself. Depending on what you're actually trying to do, there are other ways of reading an MPS file in Python, such as CyLP.
@tkralphs could you expand on these other ways please? Would they enable me to initialise a pulp LpProblem, or would I need to use another api such as mip?
I was thinking of having a look at pysmps and seeing how difficult it would be to read in and initialise a pulp LpProblem myself
I actually adapted the pysmps code to pulp (as can be seen in https://github.com/coin-or/pulp/blob/d273841cd70fdacfa00eb8d68213334e997973ce/pulp/mps_lp.py#L33 ).
This may imply that whatever is wrong in pulp, it's also in pysmps.
On Wed, Jul 7, 2021, 07:15 Christopher Parsonson @.***> wrote:
I just downloaded the MPS file, read it into Cbc without errors and solved the instance in a few seconds. So the problem is at least not with the MPS file. I'll leave it to others to determine whether the problem is with PuLP itself. Depending on what you're actually trying to do, there are other ways of reading an MPS file in Python, such as CyLP.
@tkralphs https://github.com/tkralphs could you expand on these other ways please? Would they enable me to initialise a pulp LpProblem, or would I need to use another api such as mip?
I was thinking of having a look at pysmps and seeing how difficult it would be to read in and initialise a pulp LpProblem myself
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/459#issuecomment-875287723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ42MXCOS7TMCCK77JFDTWPPH5ANCNFSM475DGCWQ .
Aside: @pchtsp Technically, pysmps
is under the MIT license and you should reproduce the copyright notice and the license if you re-use it. Since pysmps is on Pypi, though, it would seem better to fork it, improve it, and submit a pull request back to the original project. Then it can be a dependency of PuLP. It would be nice to have a stand-alone MPS reader in pure Python that all could build on, so as to avoid re-inventing the wheel. (Making a compliant and robust MPS reader is probably more difficult than it seems).
@cwfparsonson With CyLP, you could read the data from the MPS file into numpy objects and then do with that data as you wish. CyLP and PuLP can easily co-exist and share data. If you are working with MPS files, though, CyLP could be a better modeling environment overall. See here.
Thanks @tkralphs, I'll check out CyLP although if possible I would like to stick with pulp since the syntax and flexibility is so nice. I had a crack at trying to use pysmps to generate a pulp LpProblem but was unsuccessful, so for last few hours I've been trying to replace pulp with mip but mip doesn't have the flexibility I'm finding.
If anyone is able to help out with updating the pulp.LpProblem.fromMPS()
method to handle the above .mps file, that would be super helpful. Otherwise I'm going to delve into CyLP and see if I can work out how to read mps -> generate a pulp instance.
Hi,
I've been trying to interface pulp with the numpy arrays read in by CyLP. I cannot work out why pulp is still saying that the problem is infeasible. The instance initialises fine and looks correct (correct number of variables and constraints, mix of binary and continuous variable categories etc.). Is anyone more familiar with pulp able to spot something obvious which I am doing wrong?
Here is how I am reading in the mps file:
import pulp
import numpy as np
from cylp.cy import CyCoinMpsIO
class MPSLoader:
def __init__(self):
pass
def load_mps(self, path):
mps = CyCoinMpsIO()
mps.readMps(path)
return mps
def conv_sparse_to_full_matrix(self, c):
A = []
for row_idx in range(c.majorDim):
coeffs = [] # have coeff for each var
lhs_nonzero_coeffs = c.elements[c.vectorStarts[row_idx]:c.vectorStarts[row_idx+1]].tolist()
lhs_nonzero_vars = c.indices[c.vectorStarts[row_idx]:c.vectorStarts[row_idx+1]].tolist()
i = 0
coeff, var_idx = lhs_nonzero_coeffs[i], lhs_nonzero_vars[i]
for v_idx in range(c.minorDim):
if i < len(lhs_nonzero_coeffs):
# still have coeffs to add
if v_idx == lhs_nonzero_vars[i]:
# this coeff is applied to this variable
coeffs.append(lhs_nonzero_coeffs[i])
i += 1
else:
# this coeff not applied to this variable
coeffs.append(0)
else:
# this coeff not applied to this variable
coeffs.append(0)
A.append(coeffs)
return A
def load_mps_as_dict(self, path):
reader = self.load_mps(path)
attrs = ['name', 'objective_name', 'row_names', 'col_names', 'cats', 'types', 'c', 'A',
'b', 'LO', 'UP']
mps_dict = {attr: None for attr in attrs}
mps_dict['c'] = reader.objCoefficients
mps_dict['b'] = reader.rightHandSide.tolist()
mps_dict['LO'] = reader.variableLower.tolist()
mps_dict['UP'] = reader.variableUpper.tolist()
mps_dict['types'] = [chr(sign) for sign in reader.constraintSigns.tolist()]
mps_dict['A'] = self.conv_sparse_to_full_matrix(reader.matrixByRow)
mps_dict['cats'] = []
for i in reader.integerColumns:
if i == 0:
# continuous
mps_dict['cats'].append('Continuous')
elif i == 1:
# check if integer or binary
if mps_dict['LO'][i] == 0 and mps_dict['UP'][i] == 1:
# binary
mps_dict['cats'].append('Binary')
else:
# integer
mps_dict['cats'].append('Integer')
else:
raise Exception('Unrecognised integer indicator {}'.format(i))
return mps_dict
# load mps file into dict
path = '../milp/datasets/instances/1_item_placement/train/item_placement_1.mps'
mps_loader = MPSLoader()
mps_dict = mps_loader.load_mps_as_dict(path)
And this is how I then use thie mps_dict
to initialise pulp:
# initialise instance
instance = pulp.LpProblem(sense=1)
# initialise variables
variables = {f'x{i}': pulp.LpVariable(name=f'x{i}', lowBound=lb, upBound=ub, cat=cat) for i, lb, ub, cat in zip([x for x in range(1, len(mps_dict['LO']))], mps_dict['LO'], mps_dict['UP'], mps_dict['cats'])}
# initialise constraints
# collect lhs constraint values
constrs = []
A = np.array(mps_dict['A'])
for row_idx in range(len(A[:, 0])):
constr = []
for coeff, var in zip(A[row_idx, :], list(variables.values())):
constr.append(float(coeff) * var)
constrs.append(constr)
# add constraints
for i in range(len(constrs)):
if mps_dict['types'][i] == 'E':
instance += pulp.lpSum(constrs[i]) == mps_dict['b'][i]
elif mps_dict['types'][i] == 'L':
instance += pulp.lpSum(constrs[i]) <= mps_dict['b'][i]
elif mps_dict['types'][i] == 'G':
instance += pulp.lpSum(constrs[i]) >= mps_dict['b'][i]
else:
raise Exception('Unrecognised constraint type {}'.format(mps_dict['types'][i]))
# register objective function
instance += pulp.lpSum([float(coeff) * var for coeff, var in zip(mps_dict['c'], list(variables.values()))])
# solve
status = instance.solve()
print(status)
-1
Any help would be much appreciated!
Can you try generating the mps file from the pulp "instance" object in your code and compare it with the original file ? So you can see if something is lost /modified at some point during the translation ?
On Thu, Jul 8, 2021, 12:38 Christopher Parsonson @.***> wrote:
Hi,
I've been trying to interface pulp with the numpy arrays read in by CyLP. I cannot work out why pulp is still saying that the problem is infeasible. The instance initialises fine and looks correct (correct number of variables and constraints, mix of binary and continuous variable categories etc.). Is anyone more familiar with pulp able to spot something obvious which I am doing wrong?
Here is how I am reading in the mps file:
import pulpimport numpy as npfrom cylp.cy import CyCoinMpsIO class MPSLoader: def init(self): pass
def load_mps(self, path): mps = CyCoinMpsIO() mps.readMps(path) return mps def conv_sparse_to_full_matrix(self, c): A = [] for row_idx in range(c.majorDim): coeffs = [] # have coeff for each var lhs_nonzero_coeffs = c.elements[c.vectorStarts[row_idx]:c.vectorStarts[row_idx+1]].tolist() lhs_nonzero_vars = c.indices[c.vectorStarts[row_idx]:c.vectorStarts[row_idx+1]].tolist() i = 0 coeff, var_idx = lhs_nonzero_coeffs[i], lhs_nonzero_vars[i] for v_idx in range(c.minorDim): if i < len(lhs_nonzero_coeffs): # still have coeffs to add if v_idx == lhs_nonzero_vars[i]: # this coeff is applied to this variable coeffs.append(lhs_nonzero_coeffs[i]) i += 1 else: # this coeff not applied to this variable coeffs.append(0) else: # this coeff not applied to this variable coeffs.append(0) A.append(coeffs) return A def load_mps_as_dict(self, path): reader = self.load_mps(path) attrs = ['name', 'objective_name', 'row_names', 'col_names', 'cats', 'types', 'c', 'A', 'b', 'LO', 'UP'] mps_dict = {attr: None for attr in attrs} mps_dict['c'] = reader.objCoefficients mps_dict['b'] = reader.rightHandSide.tolist() mps_dict['LO'] = reader.variableLower.tolist() mps_dict['UP'] = reader.variableUpper.tolist() mps_dict['types'] = [chr(sign) for sign in reader.constraintSigns.tolist()] mps_dict['A'] = self.conv_sparse_to_full_matrix(reader.matrixByRow) mps_dict['cats'] = [] for i in reader.integerColumns: if i == 0: # continuous mps_dict['cats'].append('Continuous') elif i == 1: # check if integer or binary if mps_dict['LO'][i] == 0 and mps_dict['UP'][i] == 1: # binary mps_dict['cats'].append('Binary') else: # integer mps_dict['cats'].append('Integer') else: raise Exception('Unrecognised integer indicator {}'.format(i)) return mps_dict
load mps file into dictpath = '../milp/datasets/instances/1_item_placement/train/item_placement_1.mps'mps_loader = MPSLoader()mps_dict = mps_loader.load_mps_as_dict(path)
And this is how I then use thie mps_dict to initialise pulp:
initialise instanceinstance = pulp.LpProblem(sense=1)
initialise variablesvariables = {f'x{i}': pulp.LpVariable(name=f'x{i}', lowBound=lb, upBound=ub, cat=cat) for i, lb, ub, cat in zip([x for x in range(1, len(mps_dict['LO']))], mps_dict['LO'], mps_dict['UP'], mps_dict['cats'])}
initialise constraints# collect lhs constraint valuesconstrs = []A = np.array(mps_dict['A'])for row_idx in range(len(A[:, 0])):
constr = [] for coeff, var in zip(A[row_idx, :], list(variables.values())): constr.append(float(coeff) * var) constrs.append(constr)# add constraintsfor i in range(len(constrs)): if mps_dict['types'][i] == 'E': instance += pulp.lpSum(constrs[i]) == mps_dict['b'][i] elif mps_dict['types'][i] == 'L': instance += pulp.lpSum(constrs[i]) <= mps_dict['b'][i] elif mps_dict['types'][i] == 'G': instance += pulp.lpSum(constrs[i]) >= mps_dict['b'][i] else: raise Exception('Unrecognised constraint type {}'.format(mps_dict['types'][i])) # register objective functioninstance += pulp.lpSum([float(coeff) * var for coeff, var in zip(mps_dict['c'], list(variables.values()))])
solvestatus = instance.solve()print(status)-1
Any help would be much appreciated!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/459#issuecomment-876329771, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ47T7Z7RA4SK5X62UWDTWV5YXANCNFSM475DGCWQ .
@tkralphs , regarding adding pysmps as a dependancy: I agree it would definitely be a better option. I don't remember why I didn't do it at the time but I suspect I have a technical reason. I'll see if I can make a PR to pysmps with a refactored function that we could then use from inside pulp...
On Wed, Jul 7, 2021, 16:19 Ted Ralphs @.***> wrote:
Aside: @pchtsp https://github.com/pchtsp Technically, pysmps is under the [MIT license]( and you should reproduce the copyright notice and the license if you re-use it. Since pysmps is on Pypi, though, it would seem better to fork it, improve it, and submit a pull request back to the original project. Then it can be a dependency of PuLP. It would be nice to have a stand-alone MPS reader in pure Python that all could build on, so as to avoid re-inventing the wheel. (Making a compliant and robust MPS reader is probably more difficult than it seems).
@cwfparsonson https://github.com/cwfparsonson With CyLP, you could read the data from the MPS file into numpy objects and then do with that data as you wish. CyLP and PuLP can easily co-exist and share data. If you are working with MPS files, though, CyLP could be a better modeling environment overall. See here http://coin-or.github.io/CyLP/modules/CyCoinMpsIO.html .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/459#issuecomment-875645335, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ4ZU5VJ5MME74ANH6ULTWRO6NANCNFSM475DGCWQ .
Can you try generating the mps file from the pulp "instance" object in your code and compare it with the original file ? So you can see if something is lost /modified at some point during the translation ? … On Thu, Jul 8, 2021, 12:38 Christopher Parsonson @.**> wrote: Hi, I've been trying to interface pulp with the numpy arrays read in by CyLP. I cannot work out why pulp is still saying that the problem is infeasible. The instance initialises fine and looks correct (correct number of variables and constraints, mix of binary and continuous variable categories etc.). Is anyone more familiar with pulp able to spot something obvious which I am doing wrong? Here is how I am reading in the mps file: import pulpimport numpy as npfrom cylp.cy import CyCoinMpsIO class MPSLoader: def init(self): pass def load_mps(self, path): mps = CyCoinMpsIO() mps.readMps(path) return mps def conv_sparse_to_full_matrix(self, c): A = [] for row_idx in range(c.majorDim): coeffs = [] # have coeff for each var lhs_nonzero_coeffs = c.elements[c.vectorStarts[row_idx]:c.vectorStarts[row_idx+1]].tolist() lhs_nonzero_vars = c.indices[c.vectorStarts[row_idx]:c.vectorStarts[row_idx+1]].tolist() i = 0 coeff, var_idx = lhs_nonzero_coeffs[i], lhs_nonzero_vars[i] for v_idx in range(c.minorDim): if i < len(lhs_nonzero_coeffs): # still have coeffs to add if v_idx == lhs_nonzero_vars[i]: # this coeff is applied to this variable coeffs.append(lhs_nonzero_coeffs[i]) i += 1 else: # this coeff not applied to this variable coeffs.append(0) else: # this coeff not applied to this variable coeffs.append(0) A.append(coeffs) return A def load_mps_as_dict(self, path): reader = self.load_mps(path) attrs = ['name', 'objective_name', 'row_names', 'col_names', 'cats', 'types', 'c', 'A', 'b', 'LO', 'UP'] mps_dict = {attr: None for attr in attrs} mps_dict['c'] = reader.objCoefficients mps_dict['b'] = reader.rightHandSide.tolist() mps_dict['LO'] = reader.variableLower.tolist() mps_dict['UP'] = reader.variableUpper.tolist() mps_dict['types'] = [chr(sign) for sign in reader.constraintSigns.tolist()] mps_dict['A'] = self.conv_sparse_to_full_matrix(reader.matrixByRow) mps_dict['cats'] = [] for i in reader.integerColumns: if i == 0: # continuous mps_dict['cats'].append('Continuous') elif i == 1: # check if integer or binary if mps_dict['LO'][i] == 0 and mps_dict['UP'][i] == 1: # binary mps_dict['cats'].append('Binary') else: # integer mps_dict['cats'].append('Integer') else: raise Exception('Unrecognised integer indicator {}'.format(i)) return mps_dict # load mps file into dictpath = '../milp/datasets/instances/1_item_placement/train/item_placement_1.mps'mps_loader = MPSLoader()mps_dict = mps_loader.load_mps_as_dict(path) And this is how I then use thie mps_dict to initialise pulp: # initialise instanceinstance = pulp.LpProblem(sense=1) # initialise variablesvariables = {f'x{i}': pulp.LpVariable(name=f'x{i}', lowBound=lb, upBound=ub, cat=cat) for i, lb, ub, cat in zip([x for x in range(1, len(mps_dict['LO']))], mps_dict['LO'], mps_dict['UP'], mps_dict['cats'])} # initialise constraints# collect lhs constraint valuesconstrs = []A = np.array(mps_dict['A'])for row_idx in range(len(A[:, 0])): constr = [] for coeff, var in zip(A[row_idx, :], list(variables.values())): constr.append(float(coeff) var) constrs.append(constr)# add constraintsfor i in range(len(constrs)): if mps_dict['types'][i] == 'E': instance += pulp.lpSum(constrs[i]) == mps_dict['b'][i] elif mps_dict['types'][i] == 'L': instance += pulp.lpSum(constrs[i]) <= mps_dict['b'][i] elif mps_dict['types'][i] == 'G': instance += pulp.lpSum(constrs[i]) >= mps_dict['b'][i] else: raise Exception('Unrecognised constraint type {}'.format(mps_dict['types'][i])) # register objective functioninstance += pulp.lpSum([float(coeff) * var for coeff, var in zip(mps_dict['c'], list(variables.values()))]) # solvestatus = instance.solve()print(status)-1 Any help would be much appreciated! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#459 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ47T7Z7RA4SK5X62UWDTWV5YXANCNFSM475DGCWQ .
In my instance
object I am using CyCoinMpsIO
to load the .mps file which doesn't seem to store constraint and variable names etc so the files are difficult to compare. However, I can compare the LpProblem.writeMPS()
result with the original .mps file. It looks like the structure of the ROWS are the same, but below COLUMNS look different. There are also a total of 11,061 lines in the pulp-generated .mps file whereas in the original there are only 5,671 lines. Here are the 2 files:
Original .mps file: https://drive.google.com/file/d/1rpOx4GomiPzSry733hIXtCW1yccmG1iD/view?usp=sharing Pulp-generated mps file: https://drive.google.com/file/d/1VU2ail3NOra62J6PKFYWOLiswSgCMsOz/view?usp=sharing
Code to reproduce:
import pulp
path = '../milp/datasets/instances/1_item_placement/train/item_placement_1.mps'
variables, instance = pulp.LpProblem.fromMPS(path, sense=1)
path = '../milp/datasets/instances/1_item_placement/train/pulp_item_placement_1.mps'
instance.writeMPS(path)
Hi,
I'm wondering if the problem is more to do with pulp.LpProblem.solve()
. Here is a much simpler mps file (mip_data_set_1.mps
) so it is easy to read and compare (you should be able to copy-paste this into a text editor and save it with a .mps extension to reproduce this):
************************************************************************
*
* The data in this file represents the following problem:
*
* Minimize or maximize Z = x1 + 2x5 - x8
*
* Subject to:
*
* 2.5 <= 3x1 + x2 - 2x4 - x5 - x8
* 2x2 + 1.1x3 <= 2.1
* x3 + x6 = 4.0
* 1.8 <= 2.8x4 -1.2x7 <= 5.0
* 3.0 <= 5.6x1 + x5 + 1.9x8 <= 15.0
*
* where:
*
* 2.5 <= x1
* 0 <= x2 <= 4.1
* 0 <= x3
* 0 <= x4
* 0.5 <= x5 <= 4.0
* 0 <= x6
* 0 <= x7
* 0 <= x8 <= 4.3
*
************************************************************************
NAME EXAMPLE
ROWS
N OBJ
G ROW01
L ROW02
E ROW03
G ROW04
L ROW05
COLUMNS
COL01 OBJ 1.0
COL01 ROW01 3.0 ROW05 5.6
COL02 ROW01 1.0 ROW02 2.0
COL03 ROW02 1.1 ROW03 1.0
COL04 ROW01 -2.0 ROW04 2.8
COL05 OBJ 2.0
COL05 ROW01 -1.0 ROW05 1.0
COL06 ROW03 1.0
COL07 ROW04 -1.2
COL08 OBJ -1.0
COL08 ROW01 -1.0 ROW05 1.9
RHS
RHS1 ROW01 2.5
RHS1 ROW02 2.1
RHS1 ROW03 4.0
RHS1 ROW04 1.8
RHS1 ROW05 15.0
RANGES
RNG1 ROW04 3.2
RNG1 ROW05 12.0
BOUNDS
LO BND1 COL01 2.5
UP BND1 COL02 4.1
LO BND1 COL05 0.5
UP BND1 COL05 4.0
UP BND1 COL08 4.3
ENDATA
Trying to use pulp.LpProblem.fromMPS()
results in the following error:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-28-ed25b4b6728c> in <module>
2
3 path = 'mip_data_set_1.mps'
----> 4 variables, instance = pulp.LpProblem.fromMPS(path, sense=1)
/scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/pulp.py in fromMPS(cls, filename, sense, **kwargs)
1375 @classmethod
1376 def fromMPS(cls, filename, sense=0, **kwargs):
-> 1377 data = mpslp.readMPS(filename, sense=sense, **kwargs)
1378 return cls.fromDict(data)
1379
/scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/mps_lp.py in readMPS(path, sense, dropConsNames)
117 readMPSSetRhs(line, constraints)
118 elif mode == CORE_FILE_RHS_MODE_NO_NAME:
--> 119 readMPSSetRhs(line, constraints)
120 if line[0] not in rhs_names:
121 rhs_names.append(line[0])
/scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/mps_lp.py in readMPSSetRhs(line, constraintsDict)
168
169 def readMPSSetRhs(line, constraintsDict):
--> 170 constraintsDict[line[1]]['constant'] = - float(line[2])
171 return
172
IndexError: list index out of range
To solve this, I've implemented my own MPSLoader
using pysmps
rather than CyCoinMpsIO
so I can retain the variable and constraint names and make comparisons easier:
from pysmps import smps_loader as smps
class MPSLoader:
def __init__(self):
pass
def load_mps(self, path):
return smps.load_mps(path)
def conv_mps_to_dict(self, path):
reader = self.load_mps(path)
# init attrs
attrs = ['name', 'objective_name', 'row_names', 'col_names', 'cats', 'types', 'c', 'A', 'rhs_names',
'rhs', 'bnd_names', 'bnd']
idxs = [i for i in range(len(attrs))]
idx_to_attr = {idx: attr for idx, attr in zip(idxs, attrs)}
mps_dict = {}
for idx, attr in idx_to_attr.items():
mps_dict[attr] = reader[idx]
# reconfigure attrs for pulp
_lbs = [mps_dict['bnd'][bnd_name]['LO'] for bnd_name in mps_dict['bnd_names']]
mps_dict['LO'] = [float(item) for sublist in _lbs for item in sublist]
_ubs = [mps_dict['bnd'][bnd_name]['UP'] for bnd_name in mps_dict['bnd_names']]
mps_dict['UP'] = [float(item) for sublist in _ubs for item in sublist]
cats = []
for cat in mps_dict['cats']:
if cat == 'integral':
cats.append('Binary')
elif cat == 'continuous':
cats.append('Continuous')
elif cat == 'binary':
cats.append('Binary')
else:
raise Exception('Unrecognised variable category {}'.format(cat))
mps_dict['cats'] = cats
# collect rhs constraint values
b = []
for rhs_name in mps_dict['rhs_names']:
for coeff in mps_dict['rhs'][rhs_name]:
b.append(float(coeff))
mps_dict['b'] = b
return mps_dict
Loading this mps data into a pulp instance:
path = 'mip_data_set_1.mps'
mps_loader = MPSLoader()
mps_dict = mps_loader.conv_mps_to_dict(path)
# init problem instance
instance = pulp.LpProblem(name=mps_dict['name'], sense=1)
# init vars
variables = {name: pulp.LpVariable(name=name, lowBound=lb, upBound=ub, cat=cat) for name, lb, ub, cat in zip(mps_dict['col_names'], mps_dict['LO'], mps_dict['UP'], mps_dict['cats'])}
# init constraints
# collect lhs constraint values
constrs = []
for row_idx in range(len(mps_dict['A'][:, 0])):
constr = []
for coeff, var in zip(mps_dict['A'][row_idx, :], list(variables.values())):
constr.append(float(coeff) * var)
constrs.append(constr)
# add constraints
for i in range(len(constrs)):
if mps_dict['types'][i] == 'E':
instance += pulp.lpSum(constrs[i]) == mps_dict['b'][i]
elif mps_dict['types'][i] == 'L':
instance += pulp.lpSum(constrs[i]) <= mps_dict['b'][i]
elif mps_dict['types'][i] == 'G':
instance += pulp.lpSum(constrs[i]) >= mps_dict['b'][i]
else:
raise Exception('Unrecognised constraint type {}'.format(mps_dict['types'][i]))
# register objective function
instance += pulp.lpSum([float(coeff) * var for coeff, var in zip(mps_dict['c'], list(variables.values()))])
Calling print(instance)
seems to print out the correct formulation of the mps file:
EXAMPLE:
MINIMIZE
1.0*COL01 + 2.0*COL05 + -1.0*COL08 + 0.0
SUBJECT TO
_C1: 3 COL01 + COL02 - 2 COL04 - COL05 - COL08 >= 2.5
_C2: 2 COL02 + 1.1 COL03 <= 2.1
_C3: COL03 + COL06 = 4
_C4: 2.8 COL04 - 1.2 COL07 >= 1.8
_C5: 5.6 COL01 + COL05 + 1.9 COL08 <= 15
VARIABLES
2.5 <= COL01 <= inf Continuous
COL02 <= 4.1 Continuous
COL03 <= inf Continuous
COL04 <= inf Continuous
0.5 <= COL05 <= 4 Continuous
COL06 <= inf Continuous
COL07 <= inf Continuous
COL08 <= 4.3 Continuous
However, when I call instance.solve()
, pulp raises the following error:
---------------------------------------------------------------------------
PulpSolverError Traceback (most recent call last)
<ipython-input-27-4ad09fa8fa7d> in <module>
----> 1 instance.solve()
/scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/pulp.py in solve(self, solver, **kwargs)
1735 #time it
1736 self.solutionTime = -clock()
-> 1737 status = solver.actualSolve(self, **kwargs)
1738 self.solutionTime += clock()
1739 self.restoreObjective(wasNone, dummyVar)
/scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/apis/coin_api.py in actualSolve(self, lp, **kwargs)
99 def actualSolve(self, lp, **kwargs):
100 """Solve a well formulated lp problem"""
--> 101 return self.solve_CBC(lp, **kwargs)
102
103 def available(self):
/scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/apis/coin_api.py in solve_CBC(self, lp, use_mps)
157 pipe.close()
158 if not os.path.exists(tmpSol):
--> 159 raise PulpSolverError("Pulp: Error while executing "+self.path)
160 status, values, reducedCosts, shadowPrices, slacks, sol_status = \
161 self.readsol_MPS(tmpSol, lp, vs, variablesNames, constraintsNames)
PulpSolverError: Pulp: Error while executing /scratch/zciccwf/py36/envs/rlgnn/lib/python3.7/site-packages/pulp/apis/../solverdir/cbc/linux/64/cbc
The mps is certainly feasible, as shown by solving it with mip:
import mip
path = 'mip_data_set_1.mps'
instance = mip.Model()
instance.read(path)
status = instance.optimize(max_seconds=300)
if status == mip.OptimizationStatus.OPTIMAL:
print('optimal solution cost {} found'.format(instance.objective_value))
elif status == mip.OptimizationStatus.FEASIBLE:
print('sol.cost {} found, best possible: {}'.format(instance.objective_value, instance.objective_bound))
elif status == mip.OptimizationStatus.NO_SOLUTION_FOUND:
print('no feasible solution found, lower bound is: {}'.format(instance.objective_bound))
if status == mip.OptimizationStatus.OPTIMAL or status == mip.OptimizationStatus.FEASIBLE:
print('solution:')
for v in instance.vars:
if abs(v.x) > 1e-6: # only printing non-zeros
print('{} : {}'.format(v.name, v.x))
Output:
optimal solution cost 3.2368421052631575 found
solution:
COL01 : 2.5
COL02 : 1.05
COL04 : 0.6428571428571428
COL05 : 0.5
COL06 : 4.0
COL08 : 0.2631578947368425
Is this PulpSolveError
something which you guys have seen before? It is difficult for me to know if it is from the problem formulation or from something inside pulp which makes it think the problem is infeasible.
Here comes my analysis so far.
As always, it's a mix of things. I've only played with the last mps you shared (the small one).
Load an mps with minimization
We do support reading minimization problems. You have to fill the optional sense
argument (with pulp.LpMinimize
or pulp.LpMaximize
) in the LpProblem.fromMPS()
function.
PuLP's mps reader crashing PuLP mps reader crashes when the RANGES keyword is present (apparently we do not support it). If I take the RANGES section out, it solves (although returns a slightly different solution, as can be expected). I have no idea what the RANGES section means so I cannot be sure how to add support for it. If pysmps supports it, then it's a good excuse to try again to add it as dependency.
PuLP's PulpSolveError As part of PuLP's default CBC interface, it creates an MPS file. As I said in the previous comment, the mps file generated by pulp is not exactly the one you used to get data into pulp, so there's something that's missing (from PuLP's mps writer or from your interface code). As proof, the original mps can be given to cbc just fine:
cbc test.mps
Welcome to the CBC MILP Solver
Version: 2.10.3
Build Date: Mar 24 2020
command line - cbc /home/pchtsp/Downloads/test.mps (default strategy 1)
At line 27 NAME EXAMPLE
At line 28 ROWS
At line 35 COLUMNS
At line 47 RHS
At line 53 RANGES
At line 56 BOUNDS
At line 62 ENDATA
Problem EXAMPLE has 5 rows, 8 columns and 14 elements
Coin0008I EXAMPLE read with 0 errors
Presolve 2 (-3) rows, 3 (-5) columns and 6 (-8) elements
0 Obj 3.5 Dual inf 2.9473674 (1)
1 Obj 3.2368421
Optimal - objective value 3.2368421
After Postsolve, objective 3.2368421, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 3.236842105 - 1 iterations time 0.002, Presolve 0.00
Total time (CPU seconds): 0.00 (Wallclock seconds): 0.01
but if I generate an mps with the instance
variable from your code:
instance.writeMPS("generated_by_pulp.mps")
and then give it to cbc it throws an error:
cbc generated_by_pulp.mps
Welcome to the CBC MILP Solver
Version: 2.10.3
Build Date: Mar 24 2020
command line - cbc /home/pchtsp/Downloads/test2.mps (default strategy 1)
At line 2 NAME EXAMPLE
At line 3 ROWS
At line 10 COLUMNS
At line 28 RHS
At line 34 BOUNDS
Bad image at line 36 < UP BND COL01 inf >
Bad image at line 38 < UP BND COL03 inf >
Bad image at line 39 < UP BND COL04 inf >
Bad image at line 42 < UP BND COL06 inf >
Bad image at line 43 < UP BND COL07 inf >
At line 45 ENDATA
Problem EXAMPLE has 5 rows, 8 columns and 14 elements
Coin0008I EXAMPLE read with 5 errors
There were 5 errors on input
Total time (CPU seconds): 0.00 (Wallclock seconds): 0.00
Next time, if you want to get more information from the solver, you have to solve the problem like this:
instance.solve(pulp.PULP_CBC_CMD(msg=True))
@tkralphs I checked the pysmps project and remembered the main reason why I did not add it as dependency: they have numpy
as dependency which I find kind of hard to justify for reading mps files.
I will create an issue in their project to see if they can take it out.
@tkralphs I've opened an issue at pysmps
https://github.com/jmaerte/pysmps/issues/6 and things are moving. I hope we manage to integrate it to pulp soon.
Hi,
I am trying to read an MPS file as part of an optimisation competition (https://github.com/ds4dm/ml4co-competition). The dataset is located in instances.tar.gz (https://drive.google.com/file/d/1MytdY3IwX_aFRWdoc0mMfDN9Xg1EKUuq/view). This is quite a big data set, so here's a single 300kB .mps file from the data set which I am trying to read in as a pulp LpProblem: https://drive.google.com/file/d/1rpOx4GomiPzSry733hIXtCW1yccmG1iD/view?usp=sharing
Once downloaded, I am using the below to load the .mps instance:
This appears to read the .mps file without crashing, however it seems to load the problem as a maximisation problem when I believe it is meant to be a minimisation problem. Furthermore, when I run:
I get the following error:
Setting
sense=1
seems to prevent the above crash, however it results inpulp
saying the problem is infeasible:I do not think the problem is with the .mps file, because the following code appears to work with the
mip
library:Do you know if there might be any bugs with
pulp.LpProblem.fromMPS()
?