switch-model / switch

A Modern Platform for Planning High-Renewable Power Systems
http://switch-model.org/
Other
130 stars 84 forks source link

Resolution of Missing Upper Bound Issue When Solving With GLPK #8

Closed rhaubenstock closed 8 years ago

rhaubenstock commented 9 years ago

Reed: Hey Josiah, I just wanted to give you an update on investigating the GLPK issue.

I first ran all of the example models from the examples folder with cbc as the solver and they all worked. Then I tried to run each of them using GLPK as the solver which resulted in the models 3zone_toy, copperplate1, and discrete_build producing errors and copperplate0, custom_extension, and production_cost_models running without error.

Furthermore, the errors that resulted in running the models 3zone_toy, copperplate1, and discrete_build were all actually the same error, which is "missing upper bound". I then looked into the glpk code to see where this error message was being produced and found that it was being produced in the glpcpx.c file. More specifically, it was being produced by the parsing_bounds function, and that the token that was being found after the "<=" / " less than or equal to" token was a symbolic name token, which doesn't trigger any of the if statements and gets thrown in to the else case which is throwing the error that we see.

I haven't been able to figure out more than that, but I will keep working on it. In all honesty I'm not sure whether I should be examining the python code or how pyomo works or how glpk works in order to find out what is going wrong, but I'll check the web to see if anything similar has happened to anyone.


Josiah: That's weird. I think the problem lies in the fuel_markets module because it only appears in the problematic examples. The strange thing is that neither of the two constraints in fuel_markets use a "<="; both are equality constraints.

I'm not sure the best way to proceed either, so checking for similar problems on the web and/or making a simple example that reproduces this error and posting it with a question on the developer's list or stackoverflow makes sense.


Reed: I haven't tried looking into the fuel_markets module, but I'll look into it when I get the chance.

I think the error is fairly simple, but I'm not sure how to fix it. So the actual error I was getting looked something like this

ERROR: "[base]/site-packages/pyomo/opt/base/solvers.py", 428, solve
    Solver (glpk) returned non-zero return code (1)
ERROR: "[base]/site-packages/pyomo/opt/base/solvers.py", 433, solve
    Solver log:
    GLPSOL: GLPK LP/MIP Solver, v4.55
    Parameter(s) specified in the command line:
     --write /var/folders/fp/86bgl9md7q90fc9jztz5m4980000gn/T/tmpfKTxhX.glpk.raw
     --wglp /var/folders/fp/86bgl9md7q90fc9jztz5m4980000gn/T/tmpA8GzVb.glpk.glp
     --cpxlp /var/folders/fp/86bgl9md7q90fc9jztz5m4980000gn/T/tmpKOUxAv.pyomo.lp
    Reading problem data from '/var/folders/fp/86bgl9md7q90fc9jztz5m4980000gn/T/tmpKOUxAv.pyomo.lp'...
    /var/folders/fp/86bgl9md7q90fc9jztz5m4980000gn/T/tmpKOUxAv.pyomo.lp:5257: missing upper bound name 
    CPLEX LP file processing error
Traceback (most recent call last):
  File "solve.py", line 30, in <module>
    results = opt.solve(switch_instance, keepfiles=False, tee=False)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyomo/opt/base/solvers.py", line 435, in solve
    "Solver (%s) did not exit normally" % self.name )
pyutilib.common._exceptions.ApplicationError: Solver (glpk) did not exit normally

So I went into the file /var/folders/fp/86bgl9md7q90fc9jztz5m4980000gn/T/tmpKOUxAv.pyomo.lp and looked at line 5257 to see what was causing the error, and sure enough the line was just 0 <= x1294 <= inf , but for some reason GLPK only seems to recognize infinity when it is preceded by a + sign or a - sign, which is kind of interesting because in the 5,256 lines before it there were plenty of +inf 's which didn't cause GLPK any problems.

Thanks for the advice regarding what to do next, and I might end up posting something on stack exchange if I can't figure it out.


Josiah: I replicated your process on copperplate1, edited that line from inf to +inf, and got glpsol to solve it, which supports your assessment:

siah-macbookpro:copperplate1 josiah$ ./solve.py 
ERROR: "[base]/site-packages/pyomo/opt/base/solvers.py", 428, solve
    Solver (glpk) returned non-zero return code (1)
ERROR: "[base]/site-packages/pyomo/opt/base/solvers.py", 433, solve
    Solver log:
    GLPSOL: GLPK LP/MIP Solver, v4.52
    Parameter(s) specified in the command line:
     --write /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmptblBpo.glpk.raw
     --wglp /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpcrjXQm.glpk.glp
     --cpxlp /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp
    Reading problem data from `/var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp'...
    /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp:337: missing upper bound
    CPLEX LP file processing error
Traceback (most recent call last):
  File "./solve.py", line 26, in <module>
    results = opt.solve(switch_instance, keepfiles=False, tee=False)
  File "/Library/Python/2.7/site-packages/pyomo/opt/base/solvers.py", line 435, in solve
    "Solver (%s) did not exit normally" % self.name )
pyutilib.common._exceptions.ApplicationError: Solver (glpk) did not exit normally
siah-macbookpro:copperplate1 josiah$ bbedit /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp
siah-macbookpro:copperplate1 josiah$ glpsol --cpxlp /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp
GLPSOL: GLPK LP/MIP Solver, v4.52
Parameter(s) specified in the command line:
 --cpxlp /var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp
Reading problem data from `/var/folders/yv/q91xdmmh8xj3z80059_xrqzr0000gn/T/tmpWmfzOA.pyomo.lp'...
52 rows, 44 columns, 111 non-zeros
342 lines were read
GLPK Simplex Optimizer, v4.52
52 rows, 44 columns, 111 non-zeros
Preprocessing...
23 rows, 27 columns, 55 non-zeros
Scaling...
 A: min|aij| =  5.586e-01  max|aij| =  4.383e+03  ratio =  7.846e+03
GM: min|aij| =  6.357e-01  max|aij| =  1.573e+00  ratio =  2.474e+00
EQ: min|aij| =  4.067e-01  max|aij| =  1.000e+00  ratio =  2.459e+00
Constructing initial basis...
Size of triangular part is 23
      0: obj =   2.029492572e+07  infeas =  7.797e+01 (0)
*     9: obj =   3.939574534e+07  infeas =  0.000e+00 (0)
*    15: obj =   2.667139398e+07  infeas =  1.339e-30 (0)
OPTIMAL LP SOLUTION FOUND
Time used:   0.0 secs
Memory used: 0.1 Mb (74014 bytes)
siah-macbookpro:copperplate1 josiah$ 

I skimmed through pyomo's interface to GLPK (pyomo/solvers/plugins/solvers/GLPK.py), but had trouble tracing down where the cpxlp file actually gets written out. I finally grep'ed their codebase for cpxlp and found pyomo/repn/plugins/cpxlp.py. Searching that file for 'inf' brought me to line 771-774, which has an if statement that translates no upper bound to " <= +inf\n" and otherwise prints the value of the upper bound. In python, positive infinity is printed as 'inf'. I edited the if statement to be:

            if vardata_ub is not None and value(vardata_ub) != float('inf'):
                output_file.write(ub_string_template % value(vardata_ub))
            else:
                output_file.write(" <= +inf\n")

and that fixed the problem.

I looked fuel_markets.py and found the source of the problem. The upper bound of fuel that you can purchase in a given supply tier for a particular price will default to infinity if no upper bound was provided - effectively giving an unlimited supply at a given price. This parameter is used as the upper bound for the decision variable FuelConsumptionByTier and is translated into a constraint. Since the upper bound was defined, it used the python depiction of its value, which comes to 'inf'. I added some logic to the lines that specify the upper bound to replace float('inf') with None, and it works now. I just pushed the fix to github.

puneetchitkara commented 4 years ago

What could be the source of this error?

Deterministic concurrent LP optimizer: primal simplex, dual simplex, and barrier Showing barrier log only...

Root barrier log...

Elapsed ordering time = 5s Ordering time: 139.02s

Barrier performed 0 iterations in 568.30 seconds Error termination

Explored 0 nodes (0 simplex iterations) in 570.04 seconds Thread count was
1 (of 12 available processors)

Solution count 0

Solve interrupted (error code 10001) Best objective -, best bound -, gap -
Traceback (most recent call last):
  File "<stdin>", line 5, in <module> File
  "C:\Users\Public\Software\lib\site-
  packages\pyomo\solvers\plugins\solvers\GUROBI_RUN.py", line 114, in
  gurobi_run
    model.optimize()
  File "model.pxi", line 833, in gurobipy.Model.optimize
gurobipy.GurobiError: Out of memory

Traceback (most recent call last): File "C:\Users\Public\Software\Scripts\switch-script.py", line 11, in load_entry_point('switch-model', 'console_scripts', 'switch')() File "c:\users\puneet chitkara\switch\switch_model\main.py", line 39, in main main() File "c:\users\puneet chitkara\switch\switch_model\solve.py", line 161, in main results = solve(instance) File "c:\users\puneet chitkara\switch\switch_model\solve.py", line 731, in solve results = model.solver_manager.solve(model, opt=model.solver, solver_args) File "C:\Users\Public\Software\lib\site-packages\pyomo\opt\parallel\async_solver.py", line 28, in solve return self.execute(*args, *kwds) File "C:\Users\Public\Software\lib\site-packages\pyomo\opt\parallel\manager.py", line 107, in execute ah = self.queue(args, kwds) File "C:\Users\Public\Software\lib\site-packages\pyomo\opt\parallel\manager.py", line 122, in queue return self._perform_queue(ah, *args, *kwds) File "C:\Users\Public\Software\lib\site-packages\pyomo\opt\parallel\local.py", line 58, in _perform_queue results = opt.solve(args, **kwds) File "C:\Users\Public\Software\lib\site-packages\pyomo\opt\base\solvers.py", line 600, in solve "Solver (%s) did not exit normally" % self.name)

josiahjohnston commented 4 years ago

Looks like you ran out of memory on that machine. Common remedies: