search

intake / python-snappy

Python bindings for the snappy google library
Other
480 stars 105 forks source link

problem with python-snappy installation on a virtualenv #64

Open aurelBZH opened 6 years ago

aurelBZH commented 6 years ago

I tried to install python-snappy on a virtualenv with python 2.7. It failed

` x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c snappy/snappymodule.cc -o build/temp.linux-x86_64-2.7/snappy/snappymodule.o cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ snappy/snappymodule.cc:31:22: fatal error: snappy-c.h: Aucun fichier ou dossier de ce type compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Failed building wheel for python-snappy Running setup.py clean for python-snappy Successfully built cyvcf2 peddy Failed to build python-snappy Installing collected packages: sqlalchemy, numpy, cyvcf2, peddy, geneimpacts, python-snappy Running setup.py install for geneimpacts ... done Running setup.py install for python-snappy ... error Complete output from command /home/aurelien.beliard/.virtualenvs/varapp/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-1c10wm/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-CzAHUl-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/aurelien.beliard/.virtualenvs/varapp/include/site/python2.7/python-snappy: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'cffi_modules' warnings.warn(msg) running install running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/snappy copying snappy/init.py -> build/lib.linux-x86_64-2.7/snappy copying snappy/snappy_cffi.py -> build/lib.linux-x86_64-2.7/snappy copying snappy/snappy.py -> build/lib.linux-x86_64-2.7/snappy copying snappy/snappy_cffi_builder.py -> build/lib.linux-x86_64-2.7/snappy copying snappy/main.py -> build/lib.linux-x86_64-2.7/snappy copying snappy/snappy_formats.py -> build/lib.linux-x86_64-2.7/snappy copying snappy/hadoop_snappy.py -> build/lib.linux-x86_64-2.7/snappy running build_ext building 'snappy._snappy' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/snappy x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c snappy/snappymodule.cc -o build/temp.linux-x86_64-2.7/snappy/snappymodule.o cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ snappy/snappymodule.cc:31:22: fatal error: snappy-c.h: Aucun fichier ou dossier de ce type compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 `

I tried to reinstall libsnappy-dev to correct it, but i have another problem I tried to install cffi but there already is a problem:

static struct module_state _state;
                              ^
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c snappy/crc32c.c -o build/temp.linux-x86_64-2.7/snappy/crc32c.o
  c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/snappy/snappymodule.o build/temp.linux-x86_64-2.7/snappy/crc32c.o -lsnappy -o build/lib.linux-x86_64-2.7/snappy/_snappy.so
  c++: error: unrecognized command line option ‘-Wdate-time’
  c++: error: unrecognized command line option ‘-fstack-protector-strong’
  c++: error: unrecognized command line option ‘-Wdate-time’
  c++: error: unrecognized command line option ‘-fstack-protector-strong’
  error: command 'c++' failed with exit status 1
martindurant commented 6 years ago

I'm afraid the error above doesn't mean much to me. The "unrecognized command line option" might mean an old gcc version?

May I suggest that the easy way to get around such issues is to install using conda rather than virtualenv.

aurelBZH commented 6 years ago

I use both. In fact the strange thing is that it work perfectly outside the virtualenv

martindurant commented 6 years ago

Perhaps check the toolchain available in your PATH within and without the virtualenv? I am shooting in the dark...

aurelBZH commented 6 years ago

well outside the virtual env i also have a problem. The install goes well but when i try python -m snappy --help i have this error from _snappy import CompressError, CompressedLengthError, \ ImportError: /home/aurelien.beliard/miniconda3/lib/python2.7/site-packages/../../libstdc++.so.6: versionCXXABI_1.3.9' not found (required by /home/aurelien.beliard/miniconda3/lib/python2.7/site-packages/../../libsnappy.so.1)`

cclauss commented 6 years ago

Can not install snappy on Travis Continuous Integration even with a simple .travis.yml:

language: python
addons:
  apt:
    packages:
    - gcc
install:
  - pip install python-snappy
script:
  - true

Output is https://travis-ci.org/cclauss/snappy-hack/builds/356157907

martindurant commented 6 years ago

@cclauss , this is not surprising, the build requires a full tool chain and the snappy dev libraries. Compare with the conda recipe

cclauss commented 6 years ago

Perfect! The following works as expected:

language: python
addons:
  apt:
    packages:
    - libsnappy-dev
install:
  - pip install python-snappy
script:
  - true

Thanks!

fireundubh commented 4 years ago

Just struggled with this for a bit.

Solution

  1. Download the Snappy source from: https://github.com/google/snappy/releases
  2. Extract the ZIP or tarball to D:\dev\snappy-1.1.8 (example used below) or wherever you want.

Manual:

  1. Download the python-snappy source from: https://github.com/andrix/python-snappy/archive/master.zip
  2. Extract the ZIP to wherever you want.
  3. Run Developer Command Prompt for VS 2019.
  4. Navigate to that folder in the virtualenv/pipenv shell.
  5. In the virtualenv/pipenv shell, run: python setup.py install build_ext -n -ID:\dev\snappy-1.1.8

Pip

In the virtualenv/pipenv shell (from Developer Command Prompt for VS 2019), you should be able to run:

  1. pip install python-snappy --global-option=build_ext --global-option=-ID:\dev\snappy-1.1.8

But I'm unable to get this method to work. It should be possible though.