search

kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.
Other
219 stars 76 forks source link

Build issue on Ubuntu 16.04 #64

Closed cubranic closed 5 years ago

cubranic commented 7 years ago

Trying to install on Ubuntu 16.04.03:

$ apt-cache install libhdf5-dev libhdf5-serial-dev
...
$ pip install bitshuffle
Collecting bitshuffle
  Using cached bitshuffle-0.3.2.tar.gz
Requirement already satisfied: numpy in /home/davor/ch_util_venv/lib/python2.7/site-packages (from bitshuffle)
Requirement already satisfied: h5py in /home/davor/ch_util_venv/lib/python2.7/site-packages (from bitshuffle)
Requirement already satisfied: Cython in /home/davor/ch_util_venv/lib/python2.7/site-packages (from bitshuffle)
Requirement already satisfied: setuptools>=0.7 in /home/davor/ch_util_venv/lib/python2.7/site-packages (from bitshuffle)
Requirement already satisfied: six in /home/davor/ch_util_venv/lib/python2.7/site-packages (from h5py->bitshuffle)
Building wheels for collected packages: bitshuffle
  Running setup.py bdist_wheel for bitshuffle ... error
  Complete output from command /home/davor/ch_util_venv/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-QVwGSd/bitshuffle/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpGNehbMpip-wheel- --python-tag cp27:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/bitshuffle
  copying bitshuffle/__init__.py -> build/lib.linux-x86_64-2.7/bitshuffle
  creating build/lib.linux-x86_64-2.7/bitshuffle/tests
  copying bitshuffle/tests/test_h5plugin.py -> build/lib.linux-x86_64-2.7/bitshuffle/tests
  copying bitshuffle/tests/test_regression.py -> build/lib.linux-x86_64-2.7/bitshuffle/tests
  copying bitshuffle/tests/test_h5filter.py -> build/lib.linux-x86_64-2.7/bitshuffle/tests
  copying bitshuffle/tests/make_regression_tdata.py -> build/lib.linux-x86_64-2.7/bitshuffle/tests
  copying bitshuffle/tests/test_ext.py -> build/lib.linux-x86_64-2.7/bitshuffle/tests
  copying bitshuffle/tests/__init__.py -> build/lib.linux-x86_64-2.7/bitshuffle/tests
  creating build/lib.linux-x86_64-2.7/bitshuffle/tests/data
  copying bitshuffle/tests/data/regression_0.2.0.h5 -> build/lib.linux-x86_64-2.7/bitshuffle/tests/data
  copying bitshuffle/tests/data/regression_0.1.4.h5 -> build/lib.linux-x86_64-2.7/bitshuffle/tests/data
  copying bitshuffle/tests/data/regression_0.2.1.h5 -> build/lib.linux-x86_64-2.7/bitshuffle/tests/data
  copying bitshuffle/tests/data/regression_0.1.3.h5 -> build/lib.linux-x86_64-2.7/bitshuffle/tests/data
  running build_ext

  #################################
  # Compiling with OpenMP support #
  #################################

  building 'bitshuffle.ext' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/bitshuffle
  creating build/temp.linux-x86_64-2.7/src
  creating build/temp.linux-x86_64-2.7/lz4
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBSHUF_VERSION_MAJOR=0 -DBSHUF_VERSION_MINOR=3 -DBSHUF_VERSION_POINT=2 -I/home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include -Isrc/ -Ilz4/ -I/usr/include/python2.7 -c bitshuffle/ext.c -o build/temp.linux-x86_64-2.7/bitshuffle/ext.o -O3 -ffast-math -march=native -std=c99 -fno-strict-aliasing -fopenmp
  In file included from /home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1809:0,
                   from /home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                   from /home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                   from bitshuffle/ext.c:535:
  /home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   #warning "Using deprecated NumPy API, disable it by " \
    ^
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBSHUF_VERSION_MAJOR=0 -DBSHUF_VERSION_MINOR=3 -DBSHUF_VERSION_POINT=2 -I/home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include -Isrc/ -Ilz4/ -I/usr/include/python2.7 -c src/bitshuffle.c -o build/temp.linux-x86_64-2.7/src/bitshuffle.o -O3 -ffast-math -march=native -std=c99 -fno-strict-aliasing -fopenmp
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBSHUF_VERSION_MAJOR=0 -DBSHUF_VERSION_MINOR=3 -DBSHUF_VERSION_POINT=2 -I/home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include -Isrc/ -Ilz4/ -I/usr/include/python2.7 -c src/bitshuffle_core.c -o build/temp.linux-x86_64-2.7/src/bitshuffle_core.o -O3 -ffast-math -march=native -std=c99 -fno-strict-aliasing -fopenmp
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBSHUF_VERSION_MAJOR=0 -DBSHUF_VERSION_MINOR=3 -DBSHUF_VERSION_POINT=2 -I/home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include -Isrc/ -Ilz4/ -I/usr/include/python2.7 -c src/iochain.c -o build/temp.linux-x86_64-2.7/src/iochain.o -O3 -ffast-math -march=native -std=c99 -fno-strict-aliasing -fopenmp
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBSHUF_VERSION_MAJOR=0 -DBSHUF_VERSION_MINOR=3 -DBSHUF_VERSION_POINT=2 -I/home/davor/ch_util_venv/local/lib/python2.7/site-packages/numpy/core/include -Isrc/ -Ilz4/ -I/usr/include/python2.7 -c lz4/lz4.c -o build/temp.linux-x86_64-2.7/lz4/lz4.o -O3 -ffast-math -march=native -std=c99 -fno-strict-aliasing -fopenmp
  x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/bitshuffle/ext.o build/temp.linux-x86_64-2.7/src/bitshuffle.o build/temp.linux-x86_64-2.7/src/bitshuffle_core.o build/temp.linux-x86_64-2.7/src/iochain.o build/temp.linux-x86_64-2.7/lz4/lz4.o -lgomp -o build/lib.linux-x86_64-2.7/bitshuffle/ext.so
  building 'bitshuffle.h5' extension
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBSHUF_VERSION_MAJOR=0 -DBSHUF_VERSION_MINOR=3 -DBSHUF_VERSION_POINT=2 -Isrc/ -Ilz4/ -I/usr/include/python2.7 -c bitshuffle/h5.c -o build/temp.linux-x86_64-2.7/bitshuffle/h5.o -O3 -ffast-math -march=native -std=c99 -fno-strict-aliasing -fopenmp
  In file included from bitshuffle/h5.c:247:0:
  src/bshuf_h5filter.h:36:18: fatal error: hdf5.h: No such file or directory
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for bitshuffle
  Running setup.py clean for bitshuffle
Failed to build bitshuffle

I do have hdf5.h in /usr/include/hdf5/serial/hdf5.h. I as able to work around this by installing with CFLAGS=-I/usr/include/hdf5/serial pip install bitshuffle, but then I ran into linking error finding -lhdf5. That's because I don't have libhdf5.so, I have libhdf5_serial.so, and I don't have libhdf5_hl.so but libhdf5_serial_hl.so (both in /usr/lib/x86_64-linux-gnu). I worked around that by simply symlinking ln -s libhdf5_serial.so libhdf5.so, etc.

kiyo-masui commented 7 years ago

Can you try the build in current master rather than the Pypi version? I think this might be fixed and I just haven't gotten around to re-releasing it yet.