Errors in Python Installation

[wielandbrendel]

I am running into a few issues when trying to install the Python version. After calling sudo python3 setup.py install I get (sorry for the long traceback):

Namespace(blas64=False, extraverbose=False, float32=False, gpu=False, int32=False, root_dir='../', scs=False)
/usr/local/lib/python3.6/dist-packages/numpy/distutils/system_info.py:636: UserWarning: 
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()
/usr/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
{'define_macros': [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)], 'libraries': ['blas', 'blas'], 'library_dirs': ['/usr/lib'], 'include_dirs': ['/usr/local/include', '/usr/include'], 'language': 'c'}
/usr/local/lib/python3.6/dist-packages/numpy/distutils/system_info.py:636: UserWarning: 
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()
{'define_macros': [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)], 'libraries': ['lapack', 'lapack', 'blas', 'blas'], 'library_dirs': ['/usr/lib'], 'language': 'c', 'include_dirs': ['/usr/local/include', '/usr/include']}
running install
running bdist_egg
running egg_info
writing superscs.egg-info/PKG-INFO
writing dependency_links to superscs.egg-info/dependency_links.txt
writing requirements to superscs.egg-info/requires.txt
writing top-level names to superscs.egg-info/top_level.txt
reading manifest file 'superscs.egg-info/SOURCES.txt'
writing manifest file 'superscs.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
##################################################
# failed to compile with OpenMP;                 #
# some sections of SCS will not be parallelized. #
##################################################
building '_superscs_direct' extension
C compiler: x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC

compile options: '-DPYTHON -DCTRLC=1 -DCOPYAMATRIX -DUSE_LAPACK=1 -DLAPACK_LIB_FOUND=1 -DSVD_ACTIVATED=1 -DLAPACK_LIB_FOUND -DNO_ATLAS_INFO=1 -DHAVE_CBLAS -DNO_ATLAS_INFO=1 -DHAVE_CBLAS -DDLONG=0 -I../ -I../include -I/usr/local/lib/python3.6/dist-packages/numpy/core/include -I../linsys -I/usr/local/include -I/usr/include -I/usr/local/include -I/usr/include -I../linsys/direct/ -I../linsys/direct/external/ -I/usr/include/python3.6m -c'
extra options: '-O3 -std=c99'
x86_64-linux-gnu-gcc: ../src/ctrlc.c
x86_64-linux-gnu-gcc: ../src/directions.c
x86_64-linux-gnu-gcc: scsmodule.c
x86_64-linux-gnu-gcc: ../src/cs.c
x86_64-linux-gnu-gcc: ../src/scs_parser.c
x86_64-linux-gnu-gcc: ../src/cones.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../src/linAlg.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../src/scs.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../src/scs_version.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../src/unit_test_util.c
x86_64-linux-gnu-gcc: ../src/util.c
x86_64-linux-gnu-gcc: ../linsys/common.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/private.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_1.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/SuiteSparse_config.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_aat.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_control.c
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_2.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_defaults.c
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_dump.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_global.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_info.c
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_order.c
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_postorder.c
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_preprocess.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_post_tree.c
x86_64-linux-gnu-gcc: ../linsys/direct/external/amd_valid.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
x86_64-linux-gnu-gcc: ../linsys/direct/external/ldl.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'

Do you have any idea what's going wrong?

[wielandbrendel]

To make this easier to reproduce, I extended your Dockerfile with the following lines:

COPY python/ /superscs/python
RUN apt-get install -y python3-pip
RUN pip3 install --upgrade pip
RUN pip3 install numpy scipy
RUN cd /superscs/python && python3 setup.py install

This basically gives the same errors as reported above.

[alphaville]

@wielandbrendel The Python interface of SuperSCS needs python2 (it has been tested with Python v2.7).

[wielandbrendel]

@alphaville Great, thanks for the quick response! Python2 solves some but not all problems: changing the lines above to

RUN apt-get install -y python-pip
RUN pip install --upgrade pip
RUN pip install numpy scipy
RUN cd /superscs/python && python setup.py install

still leaves some of the errors:

gcc: ../src/util.c
gcc: scsmodule.c
gcc: ../src/unit_test_util.c
gcc: ../linsys/direct/external/amd_postorder.c
gcc: ../linsys/direct/external/amd_dump.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
gcc: ../linsys/direct/external/amd_2.c
gcc: ../linsys/direct/external/amd_control.c
gcc: ../linsys/direct/external/SuiteSparse_config.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
gcc: ../linsys/direct/external/amd_valid.c
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'
cc1: error: argument to '-O' should be a non-negative integer, 'g', 's' or 'fast'

I'd love to help with a Python3 version but fear that my non-existent knowledge of C is not much of a help here :(.

[wmtonkin]

I think the problem here is that in python/setup.py extra_compile_args = ["-O3 -std=c99"]

Changing this to extra_compile_args = ['-O3', '-std=c99']

gets ride of the "cc1: error: argument to '-O' should be " ... errors

[alphaville]

@wmtonkin In python/setup.py, line 185, the extra compilation arguments are

extra_compile_args = [""]

which version of SuperSCS are you using? You could download the latest version.

[wmtonkin]

Hello Pantelis,

Thank you for getting back to me, I very much appreciate it.

I was building from master, commit ff0ee5b457d062d5aa0f5f2c27c384f9a161a9a2

If one runs the following from the top level "scs" dir: git diff v1.3.2..ff0ee5b457d062d5aa0f5f2c27c384f9a161a9a2 -- python/setup.py

It will show (snipped for brevity):

@@ -157,14 +182,14 @@ def install_scs(**kwargs): blas_info = kwargs['blas_info'] lapack_info = kwargs['lapack_info']

• extra_compile_args = ["-O3"]
• extra_compile_args = ["-O3 -std=c99"] library_dirs = [] extra_link_args = [] libraries = [] extra_define_macros = [] sources = ['scsmodule.c', ] + glob(os.path.join(root_dir, 'src/*.c'))
• glob(os.path.join(root_dir, 'linsys/*.c')) include_dirs = [root_dir, os.path.join(root_dir, 'include'), get_include(), os.path.join(root_dir, 'linsys')]

I believe this change was introduced in commit ef306c18094eb46fd846db72232ab4cdd54971b9

Running

git diff ef306c18094eb46fd846db72232ab4cdd54971b9^..ef306c18094eb46fd846db72232ab4cdd54971b9 -- python/setup.py

diff --git a/python/setup.py b/python/setup.py index fc98245b..f7d1aa5b 100644 --- a/python/setup.py +++ b/python/setup.py @@ -157,7 +157,7 @@ def install_scs(**kwargs): blas_info = kwargs['blas_info'] lapack_info = kwargs['lapack_info']

• extra_compile_args = ["-O3"]
• extra_compile_args = ["-O3 -std=c99"] library_dirs = [] extra_link_args = [] libraries = [] @@ -182,7 +182,7 @@ def install_scs(**kwargs): extra_link_args += blas_info.pop('extra_link_args', []) + lapack_info.pop('extra_link_args', []) extra_compile_args += blas_info.pop('extra_compile_args', []) + lapack_info.pop('extra_compile_args', []) if not args.int32:
• extra_define_macros += [('DLONG', 1)] # longs for integer type

• extra_define_macros += [('DLONG', 0)] # longs for integer type

  _superscs_direct = Extension( name='_superscs_direct',

Best Regards, Bill Tonkin

[web2webs]

I think the problem here is that in python/setup.py extra_compile_args = ["-O3 -std=c99"]

Changing this to extra_compile_args = ['-O3', '-std=c99']

gets ride of the "cc1: error: argument to '-O' should be " ... errors

Thanks, it works!