search

CAChemE / pyomo-recipes

Useful conda recipes for Pyomo and dependencies.
MIT License
6 stars 2 forks source link

glpk solver not working #14

Closed ckaldemeyer closed 8 years ago

ckaldemeyer commented 8 years ago

Hi everybody,

we have used your recipe to install Python, Pyomo and solvers in a university course and everything worked fine on all operating systems.

Now I have changed my own OS to Xubuntu 16.04 64 Bit and installed all packages on the newest Anaconda version via:

conda install pyomo pyomo.extras ipopt_bin glpk --channel cachemeorg

But now somehow I get a solver error (about a missing executable) when I execute a standard problem with glpk:

WARNING: "[base]/Pyomo-4.2.10784-py3.5.egg/pyomo/solvers/plugins/solvers/GLPK_old.py", 68, executable
Could not locate the 'glpsol' executable, which is required for solver 'glpk'
Traceback (most recent call last):

...

ApplicationError: No executable found for solver 'glpk'

Using

conda list

yields

derapproximator           0.52                    py35_10    cachemeorg
...
funcdesigner              0.5620                  py35_10    cachemeorg
...
glpk                      4.57                         10    cachemeorg
...
ipopt_bin                 3.7.1                        10    cachemeorg
... 
openopt                   0.5625                  py35_10    cachemeorg
...
pyomo                     4.2.10784               py35_10    cachemeorg
pyomo.extras              2.0                     py35_10    cachemeorg
...
pyro4                     4.41                    py35_10    cachemeorg
... 
pyutilib                  5.2.3601                py35_10    cachemeorg
...
serpent                   1.12                    py35_10    cachemeorg
setproctitle              1.1.9                   py35_10    cachemeorg
...
suds-jurko                0.6                     py35_10    cachemeorg
...

So the glpk package seems to be there.

Is this a bug or am I missing something?

Thanks in advance!

franktoffel commented 8 years ago

I am not sure, but this error is usually related to the PATH (i.e. your system does not find the glpk solver).

I would check whether glpsol appears in the PATH variable.

Please, take a look to this answer to understand better what I mean with this: http://askubuntu.com/a/141723

If you cannot manage please ask again, we will try to update the recipes.

franktoffel commented 8 years ago

Another thing you can do is install GLPK via conda-forge repositories

conda remove glpk
conda install glpk --channel conda-forge

I think @Juanlu001 and @whart222 are working with conda-forge team in order to have better packages

ckaldemeyer commented 8 years ago

Another thing you can do is install GLPK via conda-forge repositories

conda remove glpk conda install glpk --channel conda-forge

I think @Juanlu001 and @whart222 are working with conda-forge team in order to have better packages

I tried this but it leads to the same result.

Related to your post above:

I know that the solvers (glpk, cbc, gurobi,..) just have to be available on the command line and ergo have to be added to PATH. It is just weird that it had worked perfectly before and now does not on my current OS.

Meanwhile I will install the solvers just via apt-get which should work.

But did you change anything? If not it seems to be OS related..

Thanks Cord

franktoffel commented 8 years ago

If you haven't done it yet, can you please open a new terminal and type: glpk -h

Let us know if this returns you an error as well.

I believe that the recipe and glpk package were not changed: https://anaconda.org/cachemeorg/glpk/files

ckaldemeyer commented 8 years ago

It says that the command glpk is not found because it has not been installed or added to PATH (I have not installed it via apt-get yet)

ckaldemeyer commented 8 years ago

After a manual solver installation via

sudo apt-get install python-glpk
sudo apt-get install glpk-utils

it works now.

Weird though that it did not with conda install..

franktoffel commented 8 years ago

Edited: In order to add the the glpk directory to the PATH manually you can take a close look to the comments like this one: On Xbunutu .profile isn't be executed so I put it in .bashrc and it works

ckaldemeyer commented 8 years ago

I know how to do that and as I said it is working fine now. But with a manual solver installation on the terminal and not with the conda install command and your packages.

It was just confusing that I did not get any error messages concerning the obviously not correctly installed glpk package.

ckaldemeyer commented 8 years ago

Before, everything worked with the conda install one-liner as described in your README

astrojuanlu commented 8 years ago

I recall having problems with glpk specifically in the past, let me see if I can understand what's happening. @ckaldemeyer is right in that the installation procedure should work as it is described in the README.

franktoffel commented 8 years ago

I believe that problem was with windows, @Juanlu001 https://github.com/CAChemE/pyomo-recipes/commits/50657109d8e5767389c80267c3823e5d417e831c/glpk

Not sure what could be the issue. Thanks @ckaldemeyer for reporting

astrojuanlu commented 8 years ago

You are right @franktoffel, that was a different kind of problem.

@ckaldemeyer, I am unable to reproduce your issue in my Linux box. I have checked that the package hasn't changed, and in fact from your first comment I can see you have the same version that is been available for more than two months now:

https://anaconda.org/cachemeorg/glpk/files

Unfortunately installing glpk using apt-get shadows the issue with the conda package. We will need some more information to identify the problem, would you please post the output of these commands?

$ conda info -a
$ conda list glpk  # Just to make sure
$ dirname $(which python)
$ ls $(!!)  # Are the files there?
$ $(dirname $(which python))/glpsol --help  # If so, this should work

Hopefully we can fix the problem!

ckaldemeyer commented 8 years ago

Here's my output:

cord@cord-Laptop:~$ .anaconda3/bin/conda info -a
Using Anaconda Cloud api site https://api.anaconda.org
Current conda install:

             platform : linux-64
        conda version : 4.0.5
  conda-build version : 1.20.0
       python version : 3.5.1.final.0
     requests version : 2.9.1
     root environment : /home/cord/.anaconda3  (writable)
  default environment : /home/cord/.anaconda3
     envs directories : /home/cord/.anaconda3/envs
        package cache : /home/cord/.anaconda3/pkgs
         channel URLs : http://conda.anaconda.org/gurobi/linux-64/
                        http://conda.anaconda.org/gurobi/noarch/
                        https://repo.continuum.io/pkgs/free/linux-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/linux-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : /home/cord/.condarc
    is foreign system : False

# conda environments:
#
iesmo-env                /home/cord/.anaconda3/envs/iesmo-env
root                  *  /home/cord/.anaconda3

sys.version: 3.5.1 |Anaconda 4.0.0 (64-bit)| (default...
sys.prefix: /home/cord/.anaconda3
sys.executable: /home/cord/.anaconda3/bin/python3
conda location: /home/cord/.anaconda3/lib/python3.5/site-packages/conda
conda-build: /home/cord/.anaconda3/bin/conda-build
conda-convert: /home/cord/.anaconda3/bin/conda-convert
conda-develop: /home/cord/.anaconda3/bin/conda-develop
conda-env: /home/cord/.anaconda3/bin/conda-env
conda-index: /home/cord/.anaconda3/bin/conda-index
conda-inspect: /home/cord/.anaconda3/bin/conda-inspect
conda-metapackage: /home/cord/.anaconda3/bin/conda-metapackage
conda-pipbuild: /home/cord/.anaconda3/bin/conda-pipbuild
conda-server: /home/cord/.anaconda3/bin/conda-server
conda-sign: /home/cord/.anaconda3/bin/conda-sign
conda-skeleton: /home/cord/.anaconda3/bin/conda-skeleton
user site dirs: 

CIO_TEST: <not set>
CONDA_DEFAULT_ENV: <not set>
CONDA_ENVS_PATH: <not set>
LD_LIBRARY_PATH: :/opt/gurobi600/linux651/lib
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/gurobi600/linux651/bin
PYTHONHOME: <not set>
PYTHONPATH: <not set>

WARNING: could not import _license.show_info
# try:
# $ conda install -n root _license
cord@cord-Laptop:~$ .anaconda3/bin/conda list glpk
# packages in environment at /home/cord/.anaconda3:
#
glpk                      4.59                          0    conda-forge
cord@cord-Laptop:~$ dirname $(which python)
/usr/bin
cord@cord-Laptop:~$ ls $(!!)
ls $(dirname $(which python))
[                                                identify-im6                         ps2pdfwr
2to3                                             iecset                               ps2ps
2to3-2.7                                         igawk                                ps2ps2
2to3-3.5                                         ijs_pxljr                            ps2txt
411toppm                                         ilbmtoppm                            psfaddtable
aa-enabled                                       imagetops                            psfgettable
...
cord@cord-Laptop:~$ $(dirname $(which python))/glpsol --help 
Usage: /usr/bin/glpsol [options...] filename

General options:
   --mps             read LP/MIP problem in fixed MPS format
   --freemps         read LP/MIP problem in free MPS format (default)
   --lp              read LP/MIP problem in CPLEX LP format

...

See GLPK web page at <http://www.gnu.org/software/glpk/glpk.html>.

Please report bugs to <bug-glpk@gnu.org>.
cord@cord-Laptop:~$
ckaldemeyer commented 8 years ago

Note that glpsol is now working because I installed it via apt-get.

astrojuanlu commented 8 years ago

Notice that even if all your environments are in ~/.anaconda3, the Python executable lives in /usr/bin because the conda binaries are not accessible from the $PATH. I suspect that you have to source activate a conda environment and install everything inside it for this to work, so try

$ conda create -n pyomo27 python=2.7
$ source activate pyomo27
(pyomo27) $ conda install pyomo pyomo.extras ipopt_bin glpk --channel cachemeorg

and then run the script again.

ckaldemeyer commented 8 years ago

Notice that even if all your environments are in ~/.anaconda3, the Python executable lives in /usr/bin because the conda binaries are not accessible from the $PATH.

Yes, you are right. But I have not added the conda binaries to PATH because it is not recommend to do so as this might lead to conflicts with the systems Python installation(s).

Here's my output:

cord@cord-Laptop:~$ .anaconda3/bin/conda create -n pyomo27 python=2.7
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ......
Solving package specifications: .........

Package plan for installation in environment /home/cord/.anaconda3/envs/pyomo27:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-2.7.11              |                0        12.0 MB
    setuptools-20.7.0          |           py27_0         453 KB
    wheel-0.29.0               |           py27_0          81 KB
    pip-8.1.1                  |           py27_1         1.5 MB
    ------------------------------------------------------------
                                           Total:        14.1 MB

The following NEW packages will be INSTALLED:

    openssl:    1.0.2h-0     
    pip:        8.1.1-py27_1 
    python:     2.7.11-0     
    readline:   6.2-2        
    setuptools: 20.7.0-py27_0
    sqlite:     3.9.2-0      
    tk:         8.5.18-0     
    wheel:      0.29.0-py27_0
    zlib:       1.2.8-0      

Proceed ([y]/n)? y

Fetching packages ...
python-2.7.11- 100% |########################################################################| Time: 0:00:17 720.08 kB/s
setuptools-20. 100% |########################################################################| Time: 0:00:00 567.91 kB/s
wheel-0.29.0-p 100% |########################################################################| Time: 0:00:00 214.76 kB/s
pip-8.1.1-py27 100% |########################################################################| Time: 0:00:03 482.89 kB/s
Extracting packages ...
[      COMPLETE      ]|###########################################################################################| 100%
Linking packages ...
[      COMPLETE      ]|###########################################################################################| 100%
#
# To activate this environment, use:
# $ source activate pyomo27
#
# To deactivate this environment, use:
# $ source deactivate
#
cord@cord-Laptop:~$ source .anaconda3/bin/activate pyomo27
discarding /home/cord/.anaconda3/bin from PATH
prepending /home/cord/.anaconda3/envs/pyomo27/bin to PATH
(pyomo27)cord@cord-Laptop:~$ conda install pyomo pyomo.extras ipopt_bin glpk --channel cachemeorg
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ........
Solving package specifications: ....

The following specifications were found to be in conflict:
  - pip (target=pip-8.1.1-py27_1.tar.bz2) -> python 3.5*
  - pip (target=pip-8.1.1-py27_1.tar.bz2) -> setuptools
  - pip (target=pip-8.1.1-py27_1.tar.bz2) -> wheel
  - pyomo -> python 3.5*
  - pyomo -> setuptools
  - pyomo.extras -> python 3.5*
  - python 2.7*
  - setuptools (target=setuptools-20.7.0-py27_0.tar.bz2) -> python 3.5*
  - wheel (target=wheel-0.29.0-py27_0.tar.bz2) -> python 3.5*
Use "conda info <package>" to see the dependencies for each package.
(pyomo27)cord@cord-Laptop:~$ cd Programmierung/IESMO/iesmo_SS2016/01
0102_PythonBasics/    01_PythonPandasIntro/ 
(pyomo27)cord@cord-Laptop:~$ cd Programmierung/IESMO/iesmo_SS2016/01
0102_PythonBasics/    01_PythonPandasIntro/ 
(pyomo27)cord@cord-Laptop:~$ python Programmierung/IESMO/iesmo_SS2016/0
00_CourseOverview/    0102_PythonBasics/    01_PythonPandasIntro/ 02_Recap/             
(pyomo27)cord@cord-Laptop:~$ python Programmierung/IESMO/iesmo_SS2016/00_CourseOverview/example_pyomo.py 
Traceback (most recent call last):
  File "Programmierung/IESMO/iesmo_SS2016/00_CourseOverview/example_pyomo.py", line 23, in <module>
    from pyomo.environ import *
ImportError: No module named pyomo.environ
(pyomo27)cord@cord-Laptop:~$
ckaldemeyer commented 8 years ago

If I change the line to from pyomo import * the error is that there is no pyomo module. Weird..

astrojuanlu commented 8 years ago

Oops, my apologies: the pyomo packages were prepared only for Python 3.5, so python=2.7 won't work (that's why the nasty dependencies error appears)

ckaldemeyer commented 8 years ago

Works now!

cord@cord-Laptop:~$ .anaconda3/bin/conda create -n pyomo35 python=3.5
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ......
Solving package specifications: .........

Package plan for installation in environment /home/cord/.anaconda3/envs/pyomo35:

The following NEW packages will be INSTALLED:

    openssl:    1.0.2h-0     
    pip:        8.1.1-py35_1 
    python:     3.5.1-0      
    readline:   6.2-2        
    setuptools: 20.7.0-py35_0
    sqlite:     3.9.2-0      
    tk:         8.5.18-0     
    wheel:      0.29.0-py35_0
    xz:         5.0.5-1      
    zlib:       1.2.8-0      

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|###########################################################################################| 100%
#
# To activate this environment, use:
# $ source activate pyomo35
#
# To deactivate this environment, use:
# $ source deactivate
#
cord@cord-Laptop:~$ source .anaconda3/bin/activate pyomo35
discarding /home/cord/.anaconda3/bin from PATH
prepending /home/cord/.anaconda3/envs/pyomo35/bin to PATH
(pyomo35)cord@cord-Laptop:~$ conda install pyomo pyomo.extras ipopt_bin glpk --channel cachemeorg
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ........
Solving package specifications: .........

Package plan for installation in environment /home/cord/.anaconda3/envs/pyomo35:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    scipy-0.17.0               |      np111py35_2        29.5 MB

The following NEW packages will be INSTALLED:

    derapproximator: 0.52-py35_10      
    funcdesigner:    0.5620-py35_10    
    glpk:            4.57-10           
    ipopt_bin:       3.7.1-10          
    libgfortran:     3.0-0             
    mkl:             11.3.1-0          
    nose:            1.3.7-py35_0      
    numpy:           1.11.0-py35_0     
    openopt:         0.5625-py35_10    
    ply:             3.8-py35_0        
    pyomo:           4.2.10784-py35_10 
    pyomo.extras:    2.0-py35_10       
    pyro4:           4.41-py35_10      
    pyutilib:        5.2.3601-py35_10  
    pyyaml:          3.11-py35_1       
    scipy:           0.17.0-np111py35_2
    serpent:         1.12-py35_10      
    setproctitle:    1.1.9-py35_10     
    six:             1.10.0-py35_0     
    suds-jurko:      0.6-py35_10       
    yaml:            0.1.6-0           

Proceed ([y]/n)? y

Fetching packages ...
scipy-0.17.0-n 100% |########################################################################| Time: 0:00:47 649.30 kB/s
Extracting packages ...
[      COMPLETE      ]|###################################################################################################################################################################################################################################################| 100%
Linking packages ...
[      COMPLETE      ]|###################################################################################################################################################################################################################################################| 100%
(pyomo35)cord@cord-Laptop:~$ python Programmierung/IESMO/iesmo_SS2016/00_CourseOverview/example_pyomo.py 
GLPSOL: GLPK LP/MIP Solver, v4.57
Parameter(s) specified in the command line:
 --write /tmp/tmp3i33nvpt.glpk.raw --wglp /tmp/tmpyet8qarl.glpk.glp --cpxlp
 /tmp/tmp0x8xw9ip.pyomo.lp
Reading problem data from '/tmp/tmp0x8xw9ip.pyomo.lp'...
4 rows, 3 columns, 5 non-zeros
29 lines were read
Writing problem data to '/tmp/tmpyet8qarl.glpk.glp'...
21 lines were written
GLPK Simplex Optimizer, v4.57
4 rows, 3 columns, 5 non-zeros
Preprocessing...
1 row, 2 columns, 2 non-zeros
Scaling...
 A: min|aij| =  2.000e+00  max|aij| =  3.000e+00  ratio =  1.500e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1
*     0: obj =  -0.000000000e+00 inf =   0.000e+00 (2)
*     2: obj =   3.600000000e+03 inf =   0.000e+00 (0)
OPTIMAL LP SOLUTION FOUND
Time used:   0.0 secs
Memory used: 0.0 Mb (40416 bytes)
Writing basic solution to '/tmp/tmp3i33nvpt.glpk.raw'...
9 lines were written
(pyomo35)cord@cord-Laptop:~$
astrojuanlu commented 8 years ago

Phew, finally! :smile: Probably a PATH issue then, we can safely close this. Thanks for your cooperation @ckaldemeyer!

ckaldemeyer commented 8 years ago

BTW, I had glpk uninstalled before, so the solver was installed from your channels.

But I still do not understand what the reason was. Normally, the standard Anaconda env is root and it had also worked before.

Anyway, thanks!!!

whart222 commented 8 years ago

@Jaunlu001 I'm also confused what might have caused this issue. Thoughts?

astrojuanlu commented 8 years ago

I don't even know how it ever worked in the first place, since with that $PATH looks impossible to me to reach the conda installed packages. Without more information on the previous state, there's not much more I can say.

DOUBLEYANG-Github commented 6 years ago

Is there anybody who can put glpsol in the PATH variable for macOS? Thank you very much