Fail to pip install python-snappy on virtualenv

[paulduf]

Hello,

I fail to run pip install python-snappy on my virtualenv with Python 3.4.4

I built snappy with brew install snappy and also when I do pip3 install python-snappy with my standard python 3.4.4 version (no venv activated) everything works fine.

My error is

(paul) ➜  python git:(master) ✗ CPPFLAGS="-I/usr/local/include -L/usr/local/lib" pip install python-snappy

Collecting python-snappy
  Using cached python-snappy-0.5.1.tar.gz
Building wheels for collected packages: python-snappy
  Running setup.py bdist_wheel for python-snappy ... error
  Complete output from command /Users/labs/venvs/paul/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/pip-build-rcpw6591/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/tmps8cfe0d6pip-wheel- --python-tag cp34:
  WARNING: '.' not a valid package name; please use only .-separated package names in setup.py
  running bdist_wheel
  running build
  running build_py
  package init file '__init__.py' not found (or not a regular file)
  creating build
  creating build/lib.macosx-10.6-intel-3.4
  copying setup.py -> build/lib.macosx-10.6-intel-3.4
  copying snappy.py -> build/lib.macosx-10.6-intel-3.4
  copying snappy_cffi.py -> build/lib.macosx-10.6-intel-3.4
  copying test_snappy.py -> build/lib.macosx-10.6-intel-3.4
  copying test_snappy_cffi.py -> build/lib.macosx-10.6-intel-3.4
  running build_ext
  building '_snappy' extension
  creating build/temp.macosx-10.6-intel-3.4
  /usr/local/bin/clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/usr/local/include -L/usr/local/lib -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c snappymodule.cc -o build/temp.macosx-10.6-intel-3.4/snappymodule.o
  clang-5.0: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
  /usr/local/bin/clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/usr/local/include -L/usr/local/lib -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c crc32c.c -o build/temp.macosx-10.6-intel-3.4/crc32c.o
  clang-5.0: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
  /usr/local/bin/clang++ -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g -I/usr/local/include -L/usr/local/lib build/temp.macosx-10.6-intel-3.4/snappymodule.o build/temp.macosx-10.6-intel-3.4/crc32c.o -lsnappy -o build/lib.macosx-10.6-intel-3.4/_snappy.so
  ld: warning: ignoring file /usr/local/lib/libsnappy.dylib, file was built for x86_64 which is not the architecture being linked (i386): /usr/local/lib/libsnappy.dylib
  ld: in '/usr/local/lib/libunwind.dylib', file was built for x86_64 which is not the architecture being linked (i386): /usr/local/lib/libunwind.dylib for architecture i386
  clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/usr/local/bin/clang++' failed with exit status 1

  ----------------------------------------
  Failed building wheel for python-snappy
  Running setup.py clean for python-snappy
Failed to build python-snappy
Installing collected packages: python-snappy
  Running setup.py install for python-snappy ... error
    Complete output from command /Users/labs/venvs/paul/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/pip-build-rcpw6591/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/pip-nld38g3m-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/labs/venvs/paul/bin/../include/site/python3.4/python-snappy:
    WARNING: '.' not a valid package name; please use only .-separated package names in setup.py
    running install
    running build
    running build_py
    package init file '__init__.py' not found (or not a regular file)
    creating build
    creating build/lib.macosx-10.6-intel-3.4
    copying setup.py -> build/lib.macosx-10.6-intel-3.4
    copying snappy.py -> build/lib.macosx-10.6-intel-3.4
    copying snappy_cffi.py -> build/lib.macosx-10.6-intel-3.4
    copying test_snappy.py -> build/lib.macosx-10.6-intel-3.4
    copying test_snappy_cffi.py -> build/lib.macosx-10.6-intel-3.4
    running build_ext
    building '_snappy' extension
    creating build/temp.macosx-10.6-intel-3.4
    /usr/local/bin/clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/usr/local/include -L/usr/local/lib -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c snappymodule.cc -o build/temp.macosx-10.6-intel-3.4/snappymodule.o
    clang-5.0: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
    /usr/local/bin/clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/usr/local/include -L/usr/local/lib -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c crc32c.c -o build/temp.macosx-10.6-intel-3.4/crc32c.o
    clang-5.0: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
    /usr/local/bin/clang++ -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g -I/usr/local/include -L/usr/local/lib build/temp.macosx-10.6-intel-3.4/snappymodule.o build/temp.macosx-10.6-intel-3.4/crc32c.o -lsnappy -o build/lib.macosx-10.6-intel-3.4/_snappy.so
    ld: warning: ignoring file /usr/local/lib/libsnappy.dylib, file was built for x86_64 which is not the architecture being linked (i386): /usr/local/lib/libsnappy.dylib
    ld: in '/usr/local/lib/libunwind.dylib', file was built for x86_64 which is not the architecture being linked (i386): /usr/local/lib/libunwind.dylib for architecture i386
    clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command '/usr/local/bin/clang++' failed with exit status 1

    ----------------------------------------
Command "/Users/labs/venvs/paul/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/pip-build-rcpw6591/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/pip-nld38g3m-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/labs/venvs/paul/bin/../include/site/python3.4/python-snappy" failed with error code 1 in /private/var/folders/pf/l5_q3mn94jb9wddqtv5h23th0000gn/T/pip-build-rcpw6591/python-snappy/ 

I think my libsnappy was built for 64 bit only and I'm trying to use it with a 32-bit program but this goes beyond my understanding. I tried brew install snappy --universal as I read some similar issues for pip installing other programs but it failed again.

I googled it and there's nothing about such a case.

Many thanks for reading my issue.

[Siecje]

I'm using Python 3.5.2 and I get this error:

$ pip install python-snappy
Collecting python-snappy
  Using cached python-snappy-0.5.1.tar.gz
Building wheels for collected packages: python-snappy
  Running setup.py bdist_wheel for python-snappy ... error
  Complete output from command /home/siecje/.virtualenvs/py3/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qsqcisp8/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpr5hhxlw8pip-wheel- --python-tag cp35:
  WARNING: '.' not a valid package name; please use only .-separated package names in setup.py
  running bdist_wheel
  running build
  running build_py
  package init file '__init__.py' not found (or not a regular file)
  creating build
  creating build/lib.linux-x86_64-3.5
  copying setup.py -> build/lib.linux-x86_64-3.5
  copying test_snappy_cffi.py -> build/lib.linux-x86_64-3.5
  copying snappy_cffi.py -> build/lib.linux-x86_64-3.5
  copying snappy.py -> build/lib.linux-x86_64-3.5
  copying test_snappy.py -> build/lib.linux-x86_64-3.5
  running build_ext
  building '_snappy' extension
  creating build/temp.linux-x86_64-3.5
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -I/home/siecje/.virtualenvs/py3/include/python3.5m -c snappymodule.cc -o build/temp.linux-x86_64-3.5/snappymodule.o
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  snappymodule.cc:31:22: fatal error: snappy-c.h: No such file or directory
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for python-snappy
  Running setup.py clean for python-snappy
Failed to build python-snappy
Installing collected packages: python-snappy
  Running setup.py install for python-snappy ... error
    Complete output from command /home/siecje/.virtualenvs/py3/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qsqcisp8/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-9nufx2eb-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/siecje/.virtualenvs/py3/include/site/python3.5/python-snappy:
    WARNING: '.' not a valid package name; please use only .-separated package names in setup.py
    running install
    running build
    running build_py
    package init file '__init__.py' not found (or not a regular file)
    creating build
    creating build/lib.linux-x86_64-3.5
    copying setup.py -> build/lib.linux-x86_64-3.5
    copying test_snappy_cffi.py -> build/lib.linux-x86_64-3.5
    copying snappy_cffi.py -> build/lib.linux-x86_64-3.5
    copying snappy.py -> build/lib.linux-x86_64-3.5
    copying test_snappy.py -> build/lib.linux-x86_64-3.5
    running build_ext
    building '_snappy' extension
    creating build/temp.linux-x86_64-3.5
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -I/home/siecje/.virtualenvs/py3/include/python3.5m -c snappymodule.cc -o build/temp.linux-x86_64-3.5/snappymodule.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    snappymodule.cc:31:22: fatal error: snappy-c.h: No such file or directory
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/home/siecje/.virtualenvs/py3/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qsqcisp8/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-9nufx2eb-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/siecje/.virtualenvs/py3/include/site/python3.5/python-snappy" failed with error code 1 in /tmp/pip-build-qsqcisp8/python-snappy/

[zurk]

this answer helps me https://stackoverflow.com/a/20678150

[martindurant]

@zurk , would you be willing to contribute a PR to update the README?

[zurk]

ok, @martindurant, give me 5 mins.

[linpanusst]

@Siecje
I have a solution to your problem. You should install some support files first

sudo apt-get install libsnappy-dev
pip3 install python-snappy

And I think it will fix your problems.

[giladsh1]

on macOS execute brew install snappy and pip install snappy (not python-snappy)

[AlmogCohen]

@giladsh1 I wonder how does your recommendation to install snappy (and not pyhon-snappy) makes sense... It seems to be a completely different python package on PyPi

[cyzanfar]

sudo yum install snappy-devel
python3 -m pip install python-snappy

[Lyla-Fischer]

I just installed python-snappy on a fresh machine, and spent a fair bit of time figuring out that there is also a dependency on the python-dev package. Fixed it with sudo apt-get install python3-dev.

[Nukker]

@Siecje I have a solution to your problem. You should install some support files first

sudo apt-get install libsnappy-dev
pip3 install python-snappy

And I think it will fix your problems.

useful

[chris1248]

Still open and not fixed. It seems that this lib is barely functioning, and bugs just seems to be ignored and not closed. 5 years later and still not closed?

[martindurant]

Yes, @chris1248 - I am sorry. I am not the author is this library, who has departed. I have just done enough to merge critical fixes and release versions that work for most people, but I do not understand the details of the build. There are lots of suggestions above of fixes for you, which would not be easy to shoehorn into a pure-pip build process, since they all seem to depend on extra dev header packages.