search

google / or-tools

Google's Operations Research tools:
https://developers.google.com/optimization/
Apache License 2.0
10.97k stars 2.1k forks source link

Unable to build Python library from source on Linux #380

Closed mxr closed 6 years ago

mxr commented 7 years ago

Hello, I'm following the instructions here to build the Python library but I'm getting a compiler error.

Repro steps:

sudo apt-get install git bison flex python-setuptools python-dev autoconf libtool zlib1g-dev texinfo help2man gawk g++ curl texlive cmake subversion
sudo apt-get install zip unzip  # I installed this to fix another error that came up

make third_party
make python

The last command fails with

g++ -fPIC -std=c++0x -fwrapv -O4 -DNDEBUG -I. -Iexamples -Iortools/gen -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DARCH_K8 -Wno-deprecated -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CBC -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CLP     -DUSE_GLOP -DUSE_BOP -I/include -I/usr/local/include/python2.7 -I/usr/lib/python2.7   -c ortools/gen/ortools/algorithms/knapsack_solver_python_wrap.cc -o objs/swig/knapsack_solver_python_wrap.o
/home/mrozentsveyg/tmp/or-tools/dependencies/install/bin/swig -I. -c++ -python  -o ortools/gen/ortools/graph/graph_python_wrap.cc -module pywrapgraph ./ortools/graph/python/graph.i
/home/mrozentsveyg/tmp/or-tools/dependencies/install/bin/swig  -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CLP -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CBC    -DUSE_GLOP -DUSE_BOP -I. -c++ -python  -o ortools/gen/ortools/linear_solver/linear_solver_python_wrap.cc -module pywraplp ./ortools/linear_solver/python/linear_solver.i
ortools/gen/ortools/algorithms/knapsack_solver_python_wrap.cc:173:21: fatal error: Python.h: No such file or directory
 # include <Python.h>
                     ^
compilation terminated.
make: *** [objs/swig/constraint_solver_python_wrap.o] Error 1

I'm on Ubuntu 14.04.5 with Python 2.7.6 and g++ version Ubuntu 4.8.4-2ubuntu1~14.04.3

Any advice is appreciated. Thanks!

lperron commented 7 years ago

can you report Makefile.local ? It looks like the python path is not found.

mxr commented 7 years ago

Here are its contents:

JDK_DIRECTORY =
UNIX_PYTHON_VER = 2.7
PATH_TO_CSHARP_COMPILER =
CLR_KEYFILE = bin/or-tools.snk

# Define UNIX_GLPK_DIR to point to a compiled version of GLPK to use it
# Define UNIX_SCIP_DIR to point to a compiled version of SCIP to use it
#   i.e.: <path>/scipoptsuite-4.0.0/scip-4.0.0
#   compile scip with GMP=false READLINE=false
# Define UNIX_GUROBI_DIR and GUROBI_LIB_VERSION to use Gurobi
# Define UNIX_CPLEX_DIR to use CPLEX

UNIX_GFLAGS_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install
UNIX_PROTOBUF_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install
UNIX_SWIG_BINARY = /home/mrozentsveyg/tmp/or-tools/dependencies/install/bin/swig
UNIX_CLP_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install
UNIX_CBC_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install

Let me know if I can give more information. Note that I was compiling on commit d2cd773edbcad7269da9cccee173161c60f79a27

mxr commented 7 years ago

Hi @lperron, I'm still unable to compile (tried again on the latest master). I added some debug code to find out about my environment but I wasn't able to get to the bottom of the issue.

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk
index 04284ce..40c588a 100755
--- a/makefiles/Makefile.python.mk
+++ b/makefiles/Makefile.python.mk
@@ -19,6 +19,12 @@ ifeq ($(shell $(PYTHON_EXECUTABLE) -c "from sys import version_info as v; print
   SWIG_PYTHON3_FLAG=-py3 -DPY3
 endif

+debug_info_python:
+       @echo $(OR_TOOLS_PYTHONPATH)
+       @echo $(PYTHONPATH)
+       @echo $(PYTHON_EXECUTABLE)
+       @echo $(UNIX_PYTHON_VER)
+
 # Main target
 CANONIC_PYTHON_EXECUTABLE = $(subst $(SPACE),$(BACKSLASH_SPACE),$(subst \,/,$(subst \\,/,$(PYTHON_EXECUTABLE))))
 ifeq ($(wildcard  $(CANONIC_PYTHON_EXECUTABLE)),)
$ make debug_info_python
/home/mrozentsveyg/tmp/or-tools:/home/mrozentsveyg/tmp/or-tools/dependencies/sources/protobuf-3.2.0/python

/usr/bin/python2.7
2.7

Is there something else I could try?

Thanks!

lperron commented 7 years ago

Try to find python.h in system paths and define ADD_PYTHON_INC in Makefile.local to tell him where to look. Alternatively, you can print PATH_TO_PYTHON_LIB and PATH_TO_PYTHON_INCLUDE.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-23 20:41 GMT+02:00 Max Rozentsveyg notifications@github.com:

Hi @lperron https://github.com/lperron, I'm still unable to compile (tried again on the latest master). I added some debug code to find out about my environment but I wasn't able to get to the bottom of the issue.

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk index 04284ce..40c588a 100755--- a/makefiles/Makefile.python.mk+++ b/makefiles/Makefile.python.mk@@ -19,6 +19,12 @@ ifeq ($(shell $(PYTHON_EXECUTABLE) -c "from sys import version_info as v; print SWIG_PYTHON3_FLAG=-py3 -DPY3 endif +debug_info_python:+ @echo $(OR_TOOLS_PYTHONPATH)+ @echo $(PYTHONPATH)+ @echo $(PYTHON_EXECUTABLE)+ @echo $(UNIX_PYTHON_VER)+

Main target

CANONIC_PYTHON_EXECUTABLE = $(subst $(SPACE),$(BACKSLASH_SPACE),$(subst \,/,$(subst \,/,$(PYTHON_EXECUTABLE)))) ifeq ($(wildcard $(CANONIC_PYTHON_EXECUTABLE)),)

$ make debug_info_python /home/mrozentsveyg/tmp/or-tools:/home/mrozentsveyg/tmp/or-tools/dependencies/sources/protobuf-3.2.0/python

/usr/bin/python2.7 2.7

Is there something else I could try?

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-303493997, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17eHGyP0o_t8AAEnE9FHdoOsaxU97ks5r8yhrgaJpZM4NdOYr .

diegoalberione commented 7 years ago

Hello, I am trying to install OrTools as the documentation says and I get the following error:

python check_python_deps.py --log=ERROR [ERROR] You are using a ortools module that doesn't have a version attribute : /root/.local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ ortools/init.pyc" Run "/usr/bin/python setup.py install --user" to upgrade. If the problem persists, remove the site-package that contains "/root/.local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/init.py c". You can do so either manually or by using pip.

I have tried with python version 2.7, 3.5 and 3.6. I use Ubuntu 14.04

I appreciate any help !!

lperron commented 7 years ago

I have built this python module on 16.04, it is unfortunately not down compatible with 14.04.

Can you build from source and create a python module?

cd temp-python27/ortools && python setup.py bdist.egg

the python egg file should be in temp-python27/ortools/ortools/dist I believe

Thanks

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-24 18:38 GMT+02:00 diegoalberione notifications@github.com:

Hello, I am trying to install OrTools as the documentation says and I get the following error:

python check_python_deps.py --log=ERROR [ERROR] You are using a ortools module that doesn't have a version attribute : /root/.local/lib/python2.7/site-packages/ortools-6.0. 4217-py2.7-linux-x86_64.egg/ ortools/init.pyc" Run "/usr/bin/python setup.py install --user" to upgrade. If the problem persists, remove the site-package that contains "/root/.local/lib/python2.7/site-packages/ortools-6.0. 4217-py2.7-linux-x86_64.egg/ortools/init.py c". You can do so either manually or by using pip.

I have tried with python version 2.7, 3.5 and 3.6. I use Ubuntu 14.04

I appreciate any help !!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-303780895, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17Vy5La_pOMJtgiPVWkb4Ek-dmTu-ks5r9F0XgaJpZM4NdOYr .

mxr commented 7 years ago

@lperron Thanks, that helped. However I ran into some more issues, with the workarounds posted below:

  1. fix_python_libraries_on_linux.sh depends on patchelf being at a hardcoded path. I fixed that by doing this before the build:
sudo apt-get install patchelf
mkdir -p dependencies/install/bin
ln -s `which patchelf` dependencies/install/bin/patchelf
  1. Compiling glog failed because of aclocal conflicts.
cd dependencies/sources/glog-0.3.5 && PATH=/home/maxr/src/or-tools/dependencies/install/bin:/home/maxr/bin:/home/maxr/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin  make install
make[1]: Entering directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5'
CDPATH="${ZSH_VERSION+.}:" && cd . && aclocal-1.14 -I m4
/bin/bash: aclocal-1.14: command not found
Makefile:957: recipe for target 'aclocal.m4' failed
make[1]: *** [aclocal.m4] Error 127
make[1]: Leaving directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5'
makefiles/Makefile.third_party.unix.mk:214: recipe for target 'dependencies/install/include/glog/logging.h' failed
make: *** [dependencies/install/include/glog/logging.h] Error 2

I worked around this by applying the below change:

--- dependencies/sources/glog-0.3.5/configure.old   2017-05-30 15:03:16.255717394 -0700
+++ dependencies/sources/glog-0.3.5/configure   2017-05-30 13:39:24.957294021 -0700
@@ -2760,7 +2760,7 @@
 # (for sanity checking)

-am__api_version='1.14'
+am__api_version='1.15'

 ac_aux_dir=
 for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do

Then ran autoreconf --force --install in the glog-0.3.5 folder, then make third_party from the root again.

  1. My Python.h path was /usr/include/python2.7 and adding the ADD_PYTHON_INC="-I/usr/include/python2.7" per your suggestion/docs unblocked me.

I was able to produce a .whl, but trying to use it showed that some modules are missing:

>>> from ortools.graph import pywrapgraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 17, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper
    return importlib.import_module('_pywrapgraph')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _pywrapgraph

Is this the same issue that you mentioned in https://github.com/google/or-tools/issues/381#issuecomment-303666990? Or related to the "ImportError: No module named pywraplp" issue at https://developers.google.com/optimization/introduction/installing?

If not, do you have advice for how to debug the missing modules?

lperron commented 7 years ago

1) patchelf should have been compiled and would have been installed prior to building the module.

2) I hate configure. I think I will try to remove my forced install of aclocal, libtool... in or-tools.

3) There are some env macros that helps debug ImportError. (see https://stackoverflow.com/questions/7332299/trace-python-imports).

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-31 2:13 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron Thanks, that helped. However I ran into some more issues, with the workarounds posted below:

  1. fix_python_libraries_on_linux.sh depends on patchelf being at a hardcoded path. I fixed that by doing this before the build:

sudo apt-get install patchelf mkdir -p dependencies/install/bin ln -s which patchelf dependencies/install/bin/patchelf

  1. Compiling glog failed because of aclocal conflicts.

cd dependencies/sources/glog-0.3.5 && PATH=/home/maxr/src/or-tools/dependencies/install/bin:/home/maxr/bin:/home/maxr/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin make install make[1]: Entering directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5' CDPATH="${ZSH_VERSION+.}:" && cd . && aclocal-1.14 -I m4 /bin/bash: aclocal-1.14: command not found Makefile:957: recipe for target 'aclocal.m4' failed make[1]: [aclocal.m4] Error 127 make[1]: Leaving directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5' makefiles/Makefile.third_party.unix.mk:214: recipe for target 'dependencies/install/include/glog/logging.h' failed make: [dependencies/install/include/glog/logging.h] Error 2

I worked around this by applying the below change:

--- dependencies/sources/glog-0.3.5/configure.old 2017-05-30 15:03:16.255717394 -0700+++ dependencies/sources/glog-0.3.5/configure 2017-05-30 13:39:24.957294021 -0700@@ -2760,7 +2760,7 @@

(for sanity checking)

-am__api_version='1.14'+am__api_version='1.15'

ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do

Then ran autoreconf --force --install in the glog-0.3.5 folder, then make third_party from the root again.

  1. My Python.h path was /usr/include/python2.7 and adding the ADD_PYTHON_INC="-I/usr/include/python2.7" per your suggestion/docs unblocked me.

I was able to produce a .whl, but trying to use it showed that some modules are missing:

from ortools.graph import pywrapgraph Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 17, in _pywrapgraph = swig_import_helper() File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper return importlib.import_module('_pywrapgraph') File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) ImportError: No module named _pywrapgraph

Is this the same issue that you mentioned in #381 (comment) https://github.com/google/or-tools/issues/381#issuecomment-303666990? Or related to the "ImportError: No module named pywraplp" issue at https://developers.google.com/optimization/introduction/installing?

If not, do you have advice for how to debug the missing modules?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305044700, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17dESeHBxGN3Ak7PbaEypzhQa1N_Cks5r_LCwgaJpZM4NdOYr .

lperron commented 7 years ago

Hi,

I just pushed a big cleanup of the make third_party, we now only build gflags, protobuf, glog and cbc. The first 3 are build using cmake, the last one is still configure.

Furthermore, I no longer build aclocal and co in dependencies, and I now rely on the system installed ones.

The new requirement is that you need swig to be installed on your platform. swig >= 2.0 is mandatory, >= 3.0 is recommended. CMake needs also to be >= 3.0.

This should simplify your process.

Tell me how it goes.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-31 7:40 GMT+02:00 Laurent Perron lperron@google.com:

1) patchelf should have been compiled and would have been installed prior to building the module.

2) I hate configure. I think I will try to remove my forced install of aclocal, libtool... in or-tools.

3) There are some env macros that helps debug ImportError. (see https://stackoverflow.com/questions/7332299/trace-python-imports).

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-31 2:13 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron Thanks, that helped. However I ran into some more issues, with the workarounds posted below:

  1. fix_python_libraries_on_linux.sh depends on patchelf being at a hardcoded path. I fixed that by doing this before the build:

sudo apt-get install patchelf mkdir -p dependencies/install/bin ln -s which patchelf dependencies/install/bin/patchelf

  1. Compiling glog failed because of aclocal conflicts.

cd dependencies/sources/glog-0.3.5 && PATH=/home/maxr/src/or-tools/dependencies/install/bin:/home/maxr/bin:/home/maxr/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin make install make[1]: Entering directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5' CDPATH="${ZSH_VERSION+.}:" && cd . && aclocal-1.14 -I m4 /bin/bash: aclocal-1.14: command not found Makefile:957: recipe for target 'aclocal.m4' failed make[1]: [aclocal.m4] Error 127 make[1]: Leaving directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5' makefiles/Makefile.third_party.unix.mk:214: recipe for target 'dependencies/install/include/glog/logging.h' failed make: [dependencies/install/include/glog/logging.h] Error 2

I worked around this by applying the below change:

--- dependencies/sources/glog-0.3.5/configure.old 2017-05-30 15:03:16.255717394 -0700+++ dependencies/sources/glog-0.3.5/configure 2017-05-30 13:39:24.957294021 -0700@@ -2760,7 +2760,7 @@

(for sanity checking)

-am__api_version='1.14'+am__api_version='1.15'

ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do

Then ran autoreconf --force --install in the glog-0.3.5 folder, then make third_party from the root again.

  1. My Python.h path was /usr/include/python2.7 and adding the ADD_PYTHON_INC="-I/usr/include/python2.7" per your suggestion/docs unblocked me.

I was able to produce a .whl, but trying to use it showed that some modules are missing:

from ortools.graph import pywrapgraph Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 17, in _pywrapgraph = swig_import_helper() File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper return importlib.import_module('_pywrapgraph') File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) ImportError: No module named _pywrapgraph

Is this the same issue that you mentioned in #381 (comment) https://github.com/google/or-tools/issues/381#issuecomment-303666990? Or related to the "ImportError: No module named pywraplp" issue at https://developers.google.com/optimization/introduction/installing?

If not, do you have advice for how to debug the missing modules?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305044700, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17dESeHBxGN3Ak7PbaEypzhQa1N_Cks5r_LCwgaJpZM4NdOYr .

mxr commented 7 years ago

@lperron thanks for the cleanup and quick response. I tried to compile but ran into an error with proto. make third_party succeeded but make python did not:

$ make python ADD_PYTHON_INC="-I/usr/include/python2.7"
cd dependencies/sources/protobuf-3.2.0/python && /usr/bin/python2.7 setup.py build
running build
running build_py
Generating google/protobuf/descriptor_pb2.py...
protoc is not installed nor found in ../src.  Please compile it or install the binary package.
makefiles/Makefile.python.mk:72: recipe for target 'dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py' failed
make: *** [dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py] Error 255

I do see protoc here:

$ find . -name protoc
./dependencies/sources/protobuf-3.2.0/cmake/build/protoc
./dependencies/install/bin/protoc

Perhaps I need to pass another flag?

lperron commented 7 years ago

OK. I need to copy protoc from ./dependencies/sources/ protobuf-3.2.0/cmake/build/protoc to ./dependencies/sources/protobuf-3.2.0/src or set the PATH to include dependencies/install/bin before generating the proto part.

To move forward, just copy the protoc binary. I will fix it tomorrow.

Thanks

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-31 21:03 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron thanks for the cleanup and quick response. I tried to compile but ran into an error with proto. make third_party succeeded but make python did not:

$ make python ADD_PYTHON_INC="-I/usr/include/python2.7" cd dependencies/sources/protobuf-3.2.0/python && /usr/bin/python2.7 setup.py build running build running build_py Generating google/protobuf/descriptor_pb2.py... protoc is not installed nor found in ../src. Please compile it or install the binary package. makefiles/Makefile.python.mk:72: recipe for target 'dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py' failed make: *** [dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py] Error 255

I do see protoc here:

$ find . -name protoc ./dependencies/sources/protobuf-3.2.0/cmake/build/protoc ./dependencies/install/bin/protoc

Perhaps I need to pass another flag?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305285548, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17djSl-zv3c6UlwWC4CXF39K27WJEks5r_bmBgaJpZM4NdOYr .

lperron commented 7 years ago

Never mind, I just pushed the fix (it was already there for windows).

Thanks for the report.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-01 0:38 GMT+02:00 Laurent Perron lperron@google.com:

OK. I need to copy protoc from ./dependencies/sources/pr otobuf-3.2.0/cmake/build/protoc to ./dependencies/sources/protobuf-3.2.0/src or set the PATH to include dependencies/install/bin before generating the proto part.

To move forward, just copy the protoc binary. I will fix it tomorrow.

Thanks

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-05-31 21:03 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron thanks for the cleanup and quick response. I tried to compile but ran into an error with proto. make third_party succeeded but make python did not:

$ make python ADD_PYTHON_INC="-I/usr/include/python2.7" cd dependencies/sources/protobuf-3.2.0/python && /usr/bin/python2.7 setup.py build running build running build_py Generating google/protobuf/descriptor_pb2.py... protoc is not installed nor found in ../src. Please compile it or install the binary package. makefiles/Makefile.python.mk:72: recipe for target 'dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py' failed make: *** [dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py] Error 255

I do see protoc here:

$ find . -name protoc ./dependencies/sources/protobuf-3.2.0/cmake/build/protoc ./dependencies/install/bin/protoc

Perhaps I need to pass another flag?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305285548, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17djSl-zv3c6UlwWC4CXF39K27WJEks5r_bmBgaJpZM4NdOYr .

mxr commented 7 years ago

Will merge master and try again. Thanks!

mxr commented 7 years ago

@lperron I was able to produce a wheel with the above directions, but using it caused some issues.

First, the build machine is able to use or-tools fine

$ gcc --version | head -n1
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
$ python -c "from ortools.graph import pywrapgraph"; echo $?
0

On the machine that I want to use or-tools (Trusty), I hit a new import issue

$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ortools.graph import pywrapgraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 28, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrapgraph', fp, pathname, description)
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /usr/local/lib/python2.7/dist-packages/ortools/graph/_pywrapgraph.so)

When the Python interpreter is started you can see it's using an older gcc (4.8.4). Also, libstd++.so.6 indeed doesn't have the required glibc:

$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX | tail -n 5
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_DEBUG_MESSAGE_LENGTH

I tried to update it using these instructions.

$ sudo apt-get install software-properties-common python-software-properties
$ sudo apt-get update
$ sudo apt-get install gcc-5 g++-5
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60 --slave /usr/bin/g++ g++ /usr/bin/g++-5

And verified the right GLIBC is in the .so file

$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX_3.4.21
GLIBCXX_3.4.21

Retrying gave a different import error:

$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ortools.graph import pywrapgraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 28, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrapgraph', fp, pathname, description)
ImportError: /usr/local/lib/python2.7/dist-packages/ortools/graph/../../ortools/libortools.so: symbol _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

Since it didn't work, I undid the glib change via

$ wget http://security.ubuntu.com/ubuntu/pool/main/g/gcc-4.8/libstdc++6_4.8.4-2ubuntu1~14.04.3_amd64.deb
$ dpkg -i libstdc++6_4.8.4-2ubuntu1~14.04.3_amd64.deb

Trying another fix, I found that I might be able to pass another CFLAG

diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk
index c0ec077..2a8db3f 100644
--- a/makefiles/Makefile.unix.mk
+++ b/makefiles/Makefile.unix.mk
@@ -248,7 +248,7 @@ endif  # MAC OS X

 CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
   -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) \
-        $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP $(GLOG_INC)
+        $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP $(GLOG_INC) -D_GLIBCXX_USE_CXX11_ABI=0

 JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \

But using wheel from that build gave the import error at the start of this post, and also seems to have broken the ortools installation on the build machine. So maybe this flag is a non-starter.

It seems like there is an incompatibility with the gcc on the desired machine & the one where the wheel was compiled. Do you have advice for ways to debug further? Is the ortools wheel not compatible with Trusty because of the glibc issue?

lperron commented 7 years ago

Let me try to summarize you build on one machine and try to use it an another machine, and you get binary incompatibilities.

If true, this is not a surprise :-(

A trick they do in manylinux wheel is that everything is compiled statically (libc and libc++)

the flags are: -static -static-libgcc.

You can try to insert this when you build the libortools and maybe the _pywwrapXXX libraries.

Anyway, and this is why I have doubt about the manylinux approach (and pypi in general). I believe it is really hard to build portable pypi modules.

To help this, we are working are to simplify the build procedure. My proposed approach is to have docker containers for you platform and build the .egg file inside docker. But we are not there at all.

Thanks

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-01 2:14 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron I was able to produce a wheel with the above directions, but using it caused some issues.

First, the build machine is able to use or-tools fine

$ gcc --version | head -n1 gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 $ python -c "from ortools.graph import pywrapgraph"; echo $? 0

On the machine that I want to use or-tools (Trusty), I hit a new import issue

$ python Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from ortools.graph import pywrapgraph Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 28, in _pywrapgraph = swig_import_helper() File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 24, in swig_import_helper _mod = imp.load_module('_pywrapgraph', fp, pathname, description) ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /usr/local/lib/python2.7/dist-packages/ortools/graph/_pywrapgraph.so)

When the Python interpreter is started you can see it's using an older gcc (4.8.4). Also, libstd++.so.6 indeed doesn't have the required glibc:

$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX | tail -n 5 GLIBCXX_3.4.16 GLIBCXX_3.4.17 GLIBCXX_3.4.18 GLIBCXX_3.4.19 GLIBCXX_DEBUG_MESSAGE_LENGTH

I tried to update it using these instructions https://gist.github.com/beci/2a2091f282042ed20cda.

$ sudo apt-get install software-properties-common python-software-properties $ sudo apt-get update $ sudo apt-get install gcc-5 g++-5 $ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60 --slave /usr/bin/g++ g++ /usr/bin/g++-5

And verified the right GLIBC is in the .so file

$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX_3.4.21 GLIBCXX_3.4.21

Retrying gave a different import error:

$ python Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from ortools.graph import pywrapgraph Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 28, in _pywrapgraph = swig_import_helper() File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 24, in swig_import_helper _mod = imp.load_module('_pywrapgraph', fp, pathname, description) ImportError: /usr/local/lib/python2.7/dist-packages/ortools/graph/../../ortools/libortools.so: symbol _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

Since it didn't work, I undid the glib change via

$ wget http://security.ubuntu.com/ubuntu/pool/main/g/gcc-4.8/libstdc++6_4.8.4-2ubuntu1~14.04.3_amd64.deb $ dpkg -i libstdc++6_4.8.4-2ubuntu1~14.04.3_amd64.deb

Trying another fix, I found that I might be able to pass another CFLAG

diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk index c0ec077..2a8db3f 100644--- a/makefiles/Makefile.unix.mk+++ b/makefiles/Makefile.unix.mk@@ -248,7 +248,7 @@ endif # MAC OS X

CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) - $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP $(GLOG_INC)+ $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP $(GLOG_INC) -D_GLIBCXX_USE_CXX11_ABI=0

JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \

But using wheel from that build gave the import error at the start of this post, and also seems to have broken the ortools installation on the build machine. So maybe this flag is a non-starter.

It seems like there is an incompatibility with the gcc on the desired machine & the wheel was compiled. Do you have advice for ways to debug further? Is the ortools wheel not compatible with Trusty because of the glibc issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305353308, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17WIQkKz0alhaUpuHyLmM5RUqg5kEks5r_gJNgaJpZM4NdOYr .

mxr commented 7 years ago

Thanks @lperron, it's starting to make more sense to me now. We are really hoping to use the wheel since we use https://github.com/jazzband/pip-tools to catch dependency conflicts, and our infra is built around pip install, but maybe we'll have to add custom code to use the egg, and we'll have to catch the dependency conflicts another way.

Can you advise where I should insert the flags? I think I understand how to do it for libortools but not the _pywrapXXX libraries

--- a/makefiles/Makefile.unix.mk
+++ b/makefiles/Makefile.unix.mk
@@ -254,4 +254,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
 JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
         -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP
 DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)
-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)
+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc
lperron commented 7 years ago

It seems good.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-01 9:26 GMT+02:00 Max Rozentsveyg notifications@github.com:

Thanks @lperron https://github.com/lperron, it's starting to make more sense to me now. We are really hoping to use the wheel since we use https://github.com/jazzband/pip-tools to catch dependency conflicts, and our infra is built around pip install, but maybe we'll have to add custom code to use the egg, and & we'll have to catch the dependency conflicts another way.

Can you advise where I should insert the flags? I think I understand how to do it for libortools but not the _pywrapXXX libraries

--- a/makefiles/Makefile.unix.mk+++ b/makefiles/Makefile.unix.mk@@ -254,4 +254,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305411688, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17Tac7OdDoggzU9qHF3FKEJ_Djqbxks5r_meRgaJpZM4NdOYr .

mxr commented 7 years ago

@lperron wanted to note that I see there were some changes recently but my branch is still against 88bf683de023c48b9b859c203fe865e411ecffc2. Trying with the below changes:

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk
index 16ae0b8..80165a8 100755
--- a/makefiles/Makefile.python.mk
+++ b/makefiles/Makefile.python.mk
@@ -316,7 +316,7 @@ else
   ifeq ($(PLATFORM),MACOSX)
        cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel
   else
-       cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg
+       cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel
   endif
 endif
        cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && twine upload dist/*
diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk
index c0ec077..04a8f7c 100644
--- a/makefiles/Makefile.unix.mk
+++ b/makefiles/Makefile.unix.mk
@@ -253,4 +253,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
 JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
         -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP
 DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)
-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)
+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc

Gave the below error

g++ -shared \
  -o lib/libortools.so \
  objs/base/bitmap.o objs/base/callback.o objs/base/file.o objs/base/filelinereader.o objs/base/join.o objs/base/mutex.o objs/base/numbers.o objs/base/random.o objs/base/recordio.o objs/base/split.o objs/base/stringpiece.o objs/base/stringprintf.o objs/base/sysinfo.o objs/base/threadpool.o objs/base/timer.o objs/base/time_support.o \
  objs/util/bitset.o objs/util/cached_log.o objs/util/fp_utils.o objs/util/graph_export.o objs/util/piecewise_linear_function.o objs/util/proto_tools.o objs/util/range_query_function.o objs/util/rational_approximation.o objs/util/rcpsp_parser.o objs/util/sorted_interval_list.o objs/util/stats.o objs/util/time_limit.o objs/util/xml_helper.o \
  objs/lp_data/lp_data.o objs/lp_data/lp_decomposer.o objs/lp_data/lp_print_utils.o objs/lp_data/lp_types.o objs/lp_data/lp_utils.o objs/lp_data/matrix_scaler.o objs/lp_data/matrix_utils.o objs/lp_data/mps_reader.o objs/lp_data/proto_utils.o objs/lp_data/sparse.o objs/lp_data/sparse_column.o \
  objs/glop/basis_representation.o objs/glop/dual_edge_norms.o objs/glop/entering_variable.o objs/glop/initial_basis.o objs/glop/lp_solver.o objs/glop/lu_factorization.o objs/glop/markowitz.o objs/glop/preprocessor.o objs/glop/primal_edge_norms.o objs/glop/reduced_costs.o objs/glop/revised_simplex.o objs/glop/status.o objs/glop/update_row.o objs/glop/variables_info.o objs/glop/variable_values.o objs/glop/parameters.pb.o \
  objs/graph/assignment.o objs/graph/astar.o objs/graph/bellman_ford.o objs/graph/cliques.o objs/graph/dijkstra.o objs/graph/linear_assignment.o objs/graph/max_flow.o objs/graph/min_cost_flow.o objs/graph/shortestpaths.o objs/graph/util.o objs/graph/flow_problem.pb.o \
  objs/algorithms/dynamic_partition.o objs/algorithms/dynamic_permutation.o objs/algorithms/find_graph_symmetries.o objs/algorithms/hungarian.o objs/algorithms/knapsack_solver.o objs/algorithms/sparse_permutation.o \
  objs/sat/boolean_problem.o objs/sat/clause.o objs/sat/cp_constraints.o objs/sat/cp_model_checker.o objs/sat/cp_model_presolve.o objs/sat/cp_model_solver.o objs/sat/cp_model_utils.o objs/sat/cumulative.o objs/sat/disjunctive.o objs/sat/drat.o objs/sat/encoding.o objs/sat/flow_costs.o objs/sat/integer.o objs/sat/integer_expr.o objs/sat/intervals.o objs/sat/linear_programming_constraint.o objs/sat/lp_utils.o objs/sat/no_cycle.o objs/sat/optimization.o objs/sat/overload_checker.o objs/sat/pb_constraint.o objs/sat/precedences.o objs/sat/sat_solver.o objs/sat/simplification.o objs/sat/symmetry.o objs/sat/table.o objs/sat/theta_tree.o objs/sat/timetable.o objs/sat/timetable_edgefinding.o objs/sat/util.o objs/sat/boolean_problem.pb.o objs/sat/cp_model.pb.o objs/sat/sat_parameters.pb.o \
  objs/bop/bop_base.o objs/bop/bop_fs.o objs/bop/bop_lns.o objs/bop/bop_ls.o objs/bop/bop_portfolio.o objs/bop/bop_solution.o objs/bop/bop_solver.o objs/bop/bop_util.o objs/bop/complete_optimizer.o objs/bop/integral_solver.o objs/bop/bop_parameters.pb.o \
  objs/linear_solver/bop_interface.o objs/linear_solver/cbc_interface.o objs/linear_solver/clp_interface.o objs/linear_solver/cplex_interface.o objs/linear_solver/glop_interface.o objs/linear_solver/glop_utils.o objs/linear_solver/glpk_interface.o objs/linear_solver/gurobi_interface.o objs/linear_solver/linear_expr.o objs/linear_solver/linear_solver.o objs/linear_solver/model_exporter.o objs/linear_solver/model_validator.o objs/linear_solver/scip_interface.o objs/linear_solver/linear_solver.pb.o \
  objs/constraint_solver/ac4_mdd_reset_table.o objs/constraint_solver/ac4r_table.o objs/constraint_solver/alldiff_cst.o objs/constraint_solver/assignment.o objs/constraint_solver/collect_variables.o objs/constraint_solver/constraints.o objs/constraint_solver/constraint_solver.o objs/constraint_solver/count_cst.o objs/constraint_solver/default_search.o objs/constraint_solver/demon_profiler.o objs/constraint_solver/deviation.o objs/constraint_solver/diffn.o objs/constraint_solver/element.o objs/constraint_solver/expr_array.o objs/constraint_solver/expr_cst.o objs/constraint_solver/expressions.o objs/constraint_solver/gcc.o objs/constraint_solver/graph_constraints.o objs/constraint_solver/hybrid.o objs/constraint_solver/interval.o objs/constraint_solver/io.o objs/constraint_solver/local_search.o objs/constraint_solver/model_cache.o objs/constraint_solver/nogoods.o objs/constraint_solver/pack.o objs/constraint_solver/range_cst.o objs/constraint_solver/resource.o objs/constraint_solver/routing.o objs/constraint_solver/routing_flags.o objs/constraint_solver/routing_neighborhoods.o objs/constraint_solver/routing_search.o objs/constraint_solver/sat_constraint.o objs/constraint_solver/sched_constraints.o objs/constraint_solver/sched_expr.o objs/constraint_solver/sched_search.o objs/constraint_solver/search.o objs/constraint_solver/softgcc.o objs/constraint_solver/table.o objs/constraint_solver/timetabling.o objs/constraint_solver/trace.o objs/constraint_solver/tree_monitor.o objs/constraint_solver/utilities.o objs/constraint_solver/visitor.o objs/constraint_solver/assignment.pb.o objs/constraint_solver/demon_profiler.pb.o objs/constraint_solver/model.pb.o objs/constraint_solver/routing_enums.pb.o objs/constraint_solver/routing_parameters.pb.o objs/constraint_solver/search_limit.pb.o objs/constraint_solver/solver_parameters.pb.o \
   /home/maxr/src/or-tools/dependencies/install/lib/libCbcSolver.a /home/maxr/src/or-tools/dependencies/install/lib/libCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libCgl.a /home/maxr/src/or-tools/dependencies/install/lib/libOsi.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiClp.a /home/maxr/src/or-tools/dependencies/install/lib/libClp.a /home/maxr/src/or-tools/dependencies/install/lib/libCoinUtils.a     /home/maxr/src/or-tools/dependencies/install/lib/libgflags.a /home/maxr/src/or-tools/dependencies/install/lib/libprotobuf.a /home/maxr/src/or-tools/dependencies/install/lib/libglog.a \
  -lz -lrt -lpthread -static -static-libgcc
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: relocation R_X86_64_32 against `__TMC_END__' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
makefiles/Makefile.cpp.mk:628: recipe for target 'lib/libortools.so' failed
make: *** [lib/libortools.so] Error 1

Should I add -fPIC to LINK_CMD in Makefile.unix.mk and retry, or another flag/config? Should I pass on the current route and try the manylinux approach you mentioned? Thanks for all the help so far!

lperron commented 7 years ago

No, it has to be done during compilation of .cc into .o. Can you dig and try to find with library (it should be a dependency) is compiled with -fPIC.

BTW, I just compile on linux, and it worked flawlessly.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-01 20:36 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron wanted to note that I see there were some changes recently but my branch is still against 88bf683 https://github.com/google/or-tools/commit/88bf683de023c48b9b859c203fe865e411ecffc2. Trying with the below changes:

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk index 16ae0b8..80165a8 100755--- a/makefiles/Makefile.python.mk+++ b/makefiles/Makefile.python.mk@@ -316,7 +316,7 @@ else ifeq ($(PLATFORM),MACOSX) cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel else- cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg+ cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel endif endif cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && twine upload dist/*diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk index c0ec077..04a8f7c 100644--- a/makefiles/Makefile.unix.mk+++ b/makefiles/Makefile.unix.mk@@ -253,4 +253,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc

Gave the below error

g++ -shared \ -o lib/libortools.so \ objs/base/bitmap.o objs/base/callback.o objs/base/file.o objs/base/filelinereader.o objs/base/join.o objs/base/mutex.o objs/base/numbers.o objs/base/random.o objs/base/recordio.o objs/base/split.o objs/base/stringpiece.o objs/base/stringprintf.o objs/base/sysinfo.o objs/base/threadpool.o objs/base/timer.o objs/base/time_support.o \ objs/util/bitset.o objs/util/cached_log.o objs/util/fp_utils.o objs/util/graph_export.o objs/util/piecewise_linear_function.o objs/util/proto_tools.o objs/util/range_query_function.o objs/util/rational_approximation.o objs/util/rcpsp_parser.o objs/util/sorted_interval_list.o objs/util/stats.o objs/util/time_limit.o objs/util/xml_helper.o \ objs/lp_data/lp_data.o objs/lp_data/lp_decomposer.o objs/lp_data/lp_print_utils.o objs/lp_data/lp_types.o objs/lp_data/lp_utils.o objs/lp_data/matrix_scaler.o objs/lp_data/matrix_utils.o objs/lp_data/mps_reader.o objs/lp_data/proto_utils.o objs/lp_data/sparse.o objs/lp_data/sparse_column.o \ objs/glop/basis_representation.o objs/glop/dual_edge_norms.o objs/glop/entering_variable.o objs/glop/initial_basis.o objs/glop/lp_solver.o objs/glop/lu_factorization.o objs/glop/markowitz.o objs/glop/preprocessor.o objs/glop/primal_edge_norms.o objs/glop/reduced_costs.o objs/glop/revised_simplex.o objs/glop/status.o objs/glop/update_row.o objs/glop/variables_info.o objs/glop/variable_values.o objs/glop/parameters.pb.o \ objs/graph/assignment.o objs/graph/astar.o objs/graph/bellman_ford.o objs/graph/cliques.o objs/graph/dijkstra.o objs/graph/linear_assignment.o objs/graph/max_flow.o objs/graph/min_cost_flow.o objs/graph/shortestpaths.o objs/graph/util.o objs/graph/flow_problem.pb.o \ objs/algorithms/dynamic_partition.o objs/algorithms/dynamic_permutation.o objs/algorithms/find_graph_symmetries.o objs/algorithms/hungarian.o objs/algorithms/knapsack_solver.o objs/algorithms/sparse_permutation.o \ objs/sat/boolean_problem.o objs/sat/clause.o objs/sat/cp_constraints.o objs/sat/cp_model_checker.o objs/sat/cp_model_presolve.o objs/sat/cp_model_solver.o objs/sat/cp_model_utils.o objs/sat/cumulative.o objs/sat/disjunctive.o objs/sat/drat.o objs/sat/encoding.o objs/sat/flow_costs.o objs/sat/integer.o objs/sat/integer_expr.o objs/sat/intervals.o objs/sat/linear_programming_constraint.o objs/sat/lp_utils.o objs/sat/no_cycle.o objs/sat/optimization.o objs/sat/overload_checker.o objs/sat/pb_constraint.o objs/sat/precedences.o objs/sat/sat_solver.o objs/sat/simplification.o objs/sat/symmetry.o objs/sat/table.o objs/sat/theta_tree.o objs/sat/timetable.o objs/sat/timetable_edgefinding.o objs/sat/util.o objs/sat/boolean_problem.pb.o objs/sat/cp_model.pb.o objs/sat/sat_parameters.pb.o \ objs/bop/bop_base.o objs/bop/bop_fs.o objs/bop/bop_lns.o objs/bop/bop_ls.o objs/bop/bop_portfolio.o objs/bop/bop_solution.o objs/bop/bop_solver.o objs/bop/bop_util.o objs/bop/complete_optimizer.o objs/bop/integral_solver.o objs/bop/bop_parameters.pb.o \ objs/linear_solver/bop_interface.o objs/linear_solver/cbc_interface.o objs/linear_solver/clp_interface.o objs/linear_solver/cplex_interface.o objs/linear_solver/glop_interface.o objs/linear_solver/glop_utils.o objs/linear_solver/glpk_interface.o objs/linear_solver/gurobi_interface.o objs/linear_solver/linear_expr.o objs/linear_solver/linear_solver.o objs/linear_solver/model_exporter.o objs/linear_solver/model_validator.o objs/linear_solver/scip_interface.o objs/linear_solver/linear_solver.pb.o \ objs/constraint_solver/ac4_mdd_reset_table.o objs/constraint_solver/ac4r_table.o objs/constraint_solver/alldiff_cst.o objs/constraint_solver/assignment.o objs/constraint_solver/collect_variables.o objs/constraint_solver/constraints.o objs/constraint_solver/constraint_solver.o objs/constraint_solver/count_cst.o objs/constraint_solver/default_search.o objs/constraint_solver/demon_profiler.o objs/constraint_solver/deviation.o objs/constraint_solver/diffn.o objs/constraint_solver/element.o objs/constraint_solver/expr_array.o objs/constraint_solver/expr_cst.o objs/constraint_solver/expressions.o objs/constraint_solver/gcc.o objs/constraint_solver/graph_constraints.o objs/constraint_solver/hybrid.o objs/constraint_solver/interval.o objs/constraint_solver/io.o objs/constraint_solver/local_search.o objs/constraint_solver/model_cache.o objs/constraint_solver/nogoods.o objs/constraint_solver/pack.o objs/constraint_solver/range_cst.o objs/constraint_solver/resource.o objs/constraint_solver/routing.o objs/constraint_solver/routing_flags.o objs/constraint_solver/routing_neighborhoods.o objs/constraint_solver/routing_search.o objs/constraint_solver/sat_constraint.o objs/constraint_solver/sched_constraints.o objs/constraint_solver/sched_expr.o objs/constraint_solver/sched_search.o objs/constraint_solver/search.o objs/constraint_solver/softgcc.o objs/constraint_solver/table.o objs/constraint_solver/timetabling.o objs/constraint_solver/trace.o objs/constraint_solver/tree_monitor.o objs/constraint_solver/utilities.o objs/constraint_solver/visitor.o objs/constraint_solver/assignment.pb.o objs/constraint_solver/demon_profiler.pb.o objs/constraint_solver/model.pb.o objs/constraint_solver/routing_enums.pb.o objs/constraint_solver/routing_parameters.pb.o objs/constraint_solver/search_limit.pb.o objs/constraint_solver/solver_parameters.pb.o \ /home/maxr/src/or-tools/dependencies/install/lib/libCbcSolver.a /home/maxr/src/or-tools/dependencies/install/lib/libCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libCgl.a /home/maxr/src/or-tools/dependencies/install/lib/libOsi.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiClp.a /home/maxr/src/or-tools/dependencies/install/lib/libClp.a /home/maxr/src/or-tools/dependencies/install/lib/libCoinUtils.a /home/maxr/src/or-tools/dependencies/install/lib/libgflags.a /home/maxr/src/or-tools/dependencies/install/lib/libprotobuf.a /home/maxr/src/or-tools/dependencies/install/lib/libglog.a \ -lz -lrt -lpthread -static -static-libgcc /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: relocation R_X86_64_32 against `__TMC_END__' can not be used when making a shared object; recompile with -fPIC /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: error adding symbols: Bad value collect2: error: ld returned 1 exit status makefiles/Makefile.cpp.mk:628: recipe for target 'lib/libortools.so' failed make: *** [lib/libortools.so] Error 1

Should I add -fPIC to LINK_CMD in Makefile.unix.mk and retry, or another flag/config? Should I pass on the current route and try the manylinux approach you mentioned? Thanks for all the help so far!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305581723, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17Y680G26DwsaUNnP9JxgZ_RrSkP2ks5r_wS0gaJpZM4NdOYr .

lperron commented 7 years ago

I take it back, glibc is not compiled with -fPIC so it does not work. shit

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-01 21:09 GMT+02:00 Laurent Perron lperron@google.com:

No, it has to be done during compilation of .cc into .o. Can you dig and try to find with library (it should be a dependency) is compiled with -fPIC.

BTW, I just compile on linux, and it worked flawlessly.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-01 20:36 GMT+02:00 Max Rozentsveyg notifications@github.com:

@lperron https://github.com/lperron wanted to note that I see there were some changes recently but my branch is still against 88bf683 https://github.com/google/or-tools/commit/88bf683de023c48b9b859c203fe865e411ecffc2. Trying with the below changes:

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk index 16ae0b8..80165a8 100755--- a/makefiles/Makefile.python.mk+++ b/makefiles/Makefile.python.mk@@ -316,7 +316,7 @@ else ifeq ($(PLATFORM),MACOSX) cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel else- cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg+ cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel endif endif cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && twine upload dist/*diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk index c0ec077..04a8f7c 100644--- a/makefiles/Makefile.unix.mk+++ b/makefiles/Makefile.unix.mk@@ -253,4 +253,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \ -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc

Gave the below error

g++ -shared \ -o lib/libortools.so \ objs/base/bitmap.o objs/base/callback.o objs/base/file.o objs/base/filelinereader.o objs/base/join.o objs/base/mutex.o objs/base/numbers.o objs/base/random.o objs/base/recordio.o objs/base/split.o objs/base/stringpiece.o objs/base/stringprintf.o objs/base/sysinfo.o objs/base/threadpool.o objs/base/timer.o objs/base/time_support.o \ objs/util/bitset.o objs/util/cached_log.o objs/util/fp_utils.o objs/util/graph_export.o objs/util/piecewise_linear_function.o objs/util/proto_tools.o objs/util/range_query_function.o objs/util/rational_approximation.o objs/util/rcpsp_parser.o objs/util/sorted_interval_list.o objs/util/stats.o objs/util/time_limit.o objs/util/xml_helper.o \ objs/lp_data/lp_data.o objs/lp_data/lp_decomposer.o objs/lp_data/lp_print_utils.o objs/lp_data/lp_types.o objs/lp_data/lp_utils.o objs/lp_data/matrix_scaler.o objs/lp_data/matrix_utils.o objs/lp_data/mps_reader.o objs/lp_data/proto_utils.o objs/lp_data/sparse.o objs/lp_data/sparse_column.o \ objs/glop/basis_representation.o objs/glop/dual_edge_norms.o objs/glop/entering_variable.o objs/glop/initial_basis.o objs/glop/lp_solver.o objs/glop/lu_factorization.o objs/glop/markowitz.o objs/glop/preprocessor.o objs/glop/primal_edge_norms.o objs/glop/reduced_costs.o objs/glop/revised_simplex.o objs/glop/status.o objs/glop/update_row.o objs/glop/variables_info.o objs/glop/variable_values.o objs/glop/parameters.pb.o \ objs/graph/assignment.o objs/graph/astar.o objs/graph/bellman_ford.o objs/graph/cliques.o objs/graph/dijkstra.o objs/graph/linear_assignment.o objs/graph/max_flow.o objs/graph/min_cost_flow.o objs/graph/shortestpaths.o objs/graph/util.o objs/graph/flow_problem.pb.o \ objs/algorithms/dynamic_partition.o objs/algorithms/dynamic_permutation.o objs/algorithms/find_graph_symmetries.o objs/algorithms/hungarian.o objs/algorithms/knapsack_solver.o objs/algorithms/sparse_permutation.o \ objs/sat/boolean_problem.o objs/sat/clause.o objs/sat/cp_constraints.o objs/sat/cp_model_checker.o objs/sat/cp_model_presolve.o objs/sat/cp_model_solver.o objs/sat/cp_model_utils.o objs/sat/cumulative.o objs/sat/disjunctive.o objs/sat/drat.o objs/sat/encoding.o objs/sat/flow_costs.o objs/sat/integer.o objs/sat/integer_expr.o objs/sat/intervals.o objs/sat/linear_programming_constraint.o objs/sat/lp_utils.o objs/sat/no_cycle.o objs/sat/optimization.o objs/sat/overload_checker.o objs/sat/pb_constraint.o objs/sat/precedences.o objs/sat/sat_solver.o objs/sat/simplification.o objs/sat/symmetry.o objs/sat/table.o objs/sat/theta_tree.o objs/sat/timetable.o objs/sat/timetable_edgefinding.o objs/sat/util.o objs/sat/boolean_problem.pb.o objs/sat/cp_model.pb.o objs/sat/sat_parameters.pb.o \ objs/bop/bop_base.o objs/bop/bop_fs.o objs/bop/bop_lns.o objs/bop/bop_ls.o objs/bop/bop_portfolio.o objs/bop/bop_solution.o objs/bop/bop_solver.o objs/bop/bop_util.o objs/bop/complete_optimizer.o objs/bop/integral_solver.o objs/bop/bop_parameters.pb.o \ objs/linear_solver/bop_interface.o objs/linear_solver/cbc_interface.o objs/linear_solver/clp_interface.o objs/linear_solver/cplex_interface.o objs/linear_solver/glop_interface.o objs/linear_solver/glop_utils.o objs/linear_solver/glpk_interface.o objs/linear_solver/gurobi_interface.o objs/linear_solver/linear_expr.o objs/linear_solver/linear_solver.o objs/linear_solver/model_exporter.o objs/linear_solver/model_validator.o objs/linear_solver/scip_interface.o objs/linear_solver/linear_solver.pb.o \ objs/constraint_solver/ac4_mdd_reset_table.o objs/constraint_solver/ac4r_table.o objs/constraint_solver/alldiff_cst.o objs/constraint_solver/assignment.o objs/constraint_solver/collect_variables.o objs/constraint_solver/constraints.o objs/constraint_solver/constraint_solver.o objs/constraint_solver/count_cst.o objs/constraint_solver/default_search.o objs/constraint_solver/demon_profiler.o objs/constraint_solver/deviation.o objs/constraint_solver/diffn.o objs/constraint_solver/element.o objs/constraint_solver/expr_array.o objs/constraint_solver/expr_cst.o objs/constraint_solver/expressions.o objs/constraint_solver/gcc.o objs/constraint_solver/graph_constraints.o objs/constraint_solver/hybrid.o objs/constraint_solver/interval.o objs/constraint_solver/io.o objs/constraint_solver/local_search.o objs/constraint_solver/model_cache.o objs/constraint_solver/nogoods.o objs/constraint_solver/pack.o objs/constraint_solver/range_cst.o objs/constraint_solver/resource.o objs/constraint_solver/routing.o objs/constraint_solver/routing_flags.o objs/constraint_solver/routing_neighborhoods.o objs/constraint_solver/routing_search.o objs/constraint_solver/sat_constraint.o objs/constraint_solver/sched_constraints.o objs/constraint_solver/sched_expr.o objs/constraint_solver/sched_search.o objs/constraint_solver/search.o objs/constraint_solver/softgcc.o objs/constraint_solver/table.o objs/constraint_solver/timetabling.o objs/constraint_solver/trace.o objs/constraint_solver/tree_monitor.o objs/constraint_solver/utilities.o objs/constraint_solver/visitor.o objs/constraint_solver/assignment.pb.o objs/constraint_solver/demon_profiler.pb.o objs/constraint_solver/model.pb.o objs/constraint_solver/routing_enums.pb.o objs/constraint_solver/routing_parameters.pb.o objs/constraint_solver/search_limit.pb.o objs/constraint_solver/solver_parameters.pb.o \ /home/maxr/src/or-tools/dependencies/install/lib/libCbcSolver.a /home/maxr/src/or-tools/dependencies/install/lib/libCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libCgl.a /home/maxr/src/or-tools/dependencies/install/lib/libOsi.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiClp.a /home/maxr/src/or-tools/dependencies/install/lib/libClp.a /home/maxr/src/or-tools/dependencies/install/lib/libCoinUtils.a /home/maxr/src/or-tools/dependencies/install/lib/libgflags.a /home/maxr/src/or-tools/dependencies/install/lib/libprotobuf.a /home/maxr/src/or-tools/dependencies/install/lib/libglog.a \ -lz -lrt -lpthread -static -static-libgcc /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: relocation R_X86_64_32 against `__TMC_END__' can not be used when making a shared object; recompile with -fPIC /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: error adding symbols: Bad value collect2: error: ld returned 1 exit status makefiles/Makefile.cpp.mk:628: recipe for target 'lib/libortools.so' failed make: *** [lib/libortools.so] Error 1

Should I add -fPIC to LINK_CMD in Makefile.unix.mk and retry, or another flag/config? Should I pass on the current route and try the manylinux approach you mentioned? Thanks for all the help so far!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305581723, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17Y680G26DwsaUNnP9JxgZ_RrSkP2ks5r_wS0gaJpZM4NdOYr .

mxr commented 7 years ago

I'm also able to compile once passing the required flags to make python, however, producing a wheel on Xenial that works on Trusty continues to have some issues.

re: egg, I tried installing the egg from pypi on Trusty and that fails with a similar error

$ easy_install ortools
Searching for ortools
Reading https://pypi.python.org/simple/ortools/
Downloading https://pypi.python.org/packages/27/49/f7546bdea4aaf05722b6a7796ee2fd69272ea869334720b04703ec29274d/ortools-6.0.4217-py2.7-linux-x86_64.egg#md5=4eb71b07792640f26b21e3470ab5c19f
Best match: ortools 6.0.4217
Processing ortools-6.0.4217-py2.7-linux-x86_64.egg
creating /usr/local/lib/service_venv/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg
Extracting ortools-6.0.4217-py2.7-linux-x86_64.egg to /usr/local/lib/service_venv/lib/python2.7/site-packages
Adding ortools 6.0.4217 to easy-install.pth file

Installed /usr/local/lib/service_venv/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg
Processing dependencies for ortools
Finished processing dependencies for ortools

$ python
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/service_venv/local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/graph/pywrapgraph.py", line 17, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/service_venv/local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper
    return importlib.import_module('_pywrapgraph')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _pywrapgraph

Is that expected? The README says Trusty is supported, but maybe that's only if you build from source?

lperron commented 7 years ago

Yes, but xenial is not down compatible with trusty. You can build on trusty and use on xenial, but not the reverse.

Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00

2017-06-02 1:59 GMT+02:00 Max Rozentsveyg notifications@github.com:

I'm also able to compile once passing the required flags to make python, however, producing a wheel on Xenial that works on Trusty continues to have some issues.

re: egg, I tried installing the egg from pypi on Trusty and that fails with a similar error

$ service_venv easy_install ortools Searching for ortools Reading https://pypi.python.org/simple/ortools/ Downloading https://pypi.python.org/packages/27/49/f7546bdea4aaf05722b6a7796ee2fd69272ea869334720b04703ec29274d/ortools-6.0.4217-py2.7-linux-x86_64.egg#md5=4eb71b07792640f26b21e3470ab5c19f Best match: ortools 6.0.4217 Processing ortools-6.0.4217-py2.7-linux-x86_64.egg creating /usr/local/lib/service_venv/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg Extracting ortools-6.0.4217-py2.7-linux-x86_64.egg to /usr/local/lib/service_venv/lib/python2.7/site-packages Adding ortools 6.0.4217 to easy-install.pth file

Installed /usr/local/lib/service_venv/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg Processing dependencies for ortools Finished processing dependencies for ortools

$ service_venv python Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/service_venv/local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/graph/pywrapgraph.py", line 17, in _pywrapgraph = swig_import_helper() File "/usr/local/lib/service_venv/local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper return importlib.import_module('_pywrapgraph') File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) ImportError: No module named _pywrapgraph

Is that expected? The README says Trusty is supported, but maybe that's only if you build from source?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/380#issuecomment-305652414, or mute the thread https://github.com/notifications/unsubscribe-auth/AKj17ZA4Foc5cEPDJXjTas-ym9FAVE4wks5r_1BTgaJpZM4NdOYr .

mxr commented 7 years ago

@lperron good news, I was able to build an egg (and a wheel!) on Trusty that didn't have the import error by running a combination of the steps above. However we are wary to use/host the wheel I built internally, because it depends on me building it locally with custom work. Specifically here is what I had to work around to build a wheel on Trusty that I could use on a different machine:

  1. I still had trouble with patchelf. I had to manually do make dependencies/install/bin/patchelf. Without this, I see errors that say dependencies/install/bin/patchelf: not found and the produces egg gives the No module named _pywrapgraph error. But an egg and wheel with this change work fine. So I think patchelf is not being built right, at least on Trusty.
  2. bdist_wheel is not an option to setup.py for Linux. I remember it was tried in d2cd773edbcad7269da9cccee173161c60f79a27 and it caused other issues. We would need to re-attempt that change.
  3. The egg on pypi is built on Xenial which isn't compatible with Trusty. We would need a build to be produced on Trusty & uploaded that to pypi.
  4. HEAD produces a build that depends on protobuf==3.3.0. We are still on 3.2.0 so being able to produce a Python library that uses >=3.2.0 would be great for us.

Would fixing these issues and uploading a wheel to pypi be on your roadmap? We can also try to contribute upstream fixes for these issues (though we might need some guidance)

Thanks!

lperron commented 7 years ago

Hi,

Please check my last mail on docker support. You can now build python2 and python3 egg and wheel file for ubuntu 14.04, 16.04 and 17.04 with little effort.

This should simplify everything a lot.

lperron commented 7 years ago

I have pushed version 6.3 with support for multiple linux distributions. I have changed the name of the modules to reflect the linux distribution. So you can install ortools-ubuntu-14.04, ortools-ubuntu-16.04 on your machines.

Thanks