[Request] submit wheel for (many)Linux to PyPI

[henry0312]

Please submit wheel for not only macos, but also linux to PyPI

because installing fastFM on Docker fails, (installing on Ubuntu and Debian works fine)

root@6ab076dfe97f:/# pip install fastFM
Collecting fastFM
  Downloading fastFM-0.2.10.tar.gz (1.6MB)
    100% |████████████████████████████████| 1.6MB 784kB/s
Requirement already satisfied: numpy in /usr/local/lib/python3.6/site-packages (from fastFM)
Collecting scikit-learn (from fastFM)
  Downloading scikit_learn-0.19.1-cp36-cp36m-manylinux1_x86_64.whl (12.4MB)
    100% |████████████████████████████████| 12.4MB 125kB/s
Requirement already satisfied: scipy in /usr/local/lib/python3.6/site-packages (from fastFM)
Requirement already satisfied: cython in /usr/local/lib/python3.6/site-packages (from fastFM)
Building wheels for collected packages: fastFM
  Running setup.py bdist_wheel for fastFM ... error
  Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ahqg4ru8/fastFM/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpwpss_4uhpip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.6
  creating build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/sgd.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/datasets.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/als.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/validation.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/utils.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/base.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/__init__.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/bpr.py -> build/lib.linux-x86_64-3.6/fastFM
  copying fastFM/mcmc.py -> build/lib.linux-x86_64-3.6/fastFM
  running build_ext
  skipping 'fastFM/ffm.c' Cython extension (up-to-date)
  building 'ffm' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/fastFM
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -IfastFM/ -IfastFM-core/include/ -IfastFM-core/externals/CXSparse/Include/ -I/usr/local/lib/python3.6/site-packages/numpy/core/include -I/usr/local/include/python3.6m -c fastFM/ffm.c -o build/temp.linux-x86_64-3.6/fastFM/ffm.o
  In file included from /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1809:0,
                   from /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                   from /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                   from fastFM/ffm.c:528:
  /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   #warning "Using deprecated NumPy API, disable it by " \
    ^
  gcc -pthread -shared build/temp.linux-x86_64-3.6/fastFM/ffm.o -LfastFM/ -LfastFM-core/bin/ -L/usr/local/lib -lm -lfastfm -lpython3.6m -o build/lib.linux-x86_64-3.6/ffm.cpython-36m-x86_64-linux-gnu.so
  /usr/bin/ld: fastFM-core/bin//libfastfm.a(ffm_utils.o): unrecognized relocation (0x2a) in section `.text'
  /usr/bin/ld: final link failed: Bad value
  collect2: error: ld returned 1 exit status
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for fastFM
  Running setup.py clean for fastFM
Failed to build fastFM
Installing collected packages: scikit-learn, fastFM
  Running setup.py install for fastFM ... error
    Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ahqg4ru8/fastFM/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-ej8i3sar-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/sgd.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/datasets.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/als.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/validation.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/utils.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/base.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/__init__.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/bpr.py -> build/lib.linux-x86_64-3.6/fastFM
    copying fastFM/mcmc.py -> build/lib.linux-x86_64-3.6/fastFM
    running build_ext
    skipping 'fastFM/ffm.c' Cython extension (up-to-date)
    building 'ffm' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/fastFM
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -IfastFM/ -IfastFM-core/include/ -IfastFM-core/externals/CXSparse/Include/ -I/usr/local/lib/python3.6/site-packages/numpy/core/include -I/usr/local/include/python3.6m -c fastFM/ffm.c -o build/temp.linux-x86_64-3.6/fastFM/ffm.o
    In file included from /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1809:0,
                     from /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                     from /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from fastFM/ffm.c:528:
    /usr/local/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     #warning "Using deprecated NumPy API, disable it by " \
      ^
    gcc -pthread -shared build/temp.linux-x86_64-3.6/fastFM/ffm.o -LfastFM/ -LfastFM-core/bin/ -L/usr/local/lib -lm -lfastfm -lpython3.6m -o build/lib.linux-x86_64-3.6/ffm.cpython-36m-x86_64-linux-gnu.so
    /usr/bin/ld: fastFM-core/bin//libfastfm.a(ffm_utils.o): unrecognized relocation (0x2a) in section `.text'
    /usr/bin/ld: final link failed: Bad value
    collect2: error: ld returned 1 exit status
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ahqg4ru8/fastFM/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-ej8i3sar-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-ahqg4ru8/fastFM/

[ibayer]

@zouzias thanks for the pointers!

Background on wheels that support multiple Linux distros. https://www.youtube.com/watch?v=oE5iePv8nD8

Project dedicated to building wheels for multiple distros. https://github.com/pypa/manylinux

Starting point for our build process. https://github.com/zouzias/pyboostrtree/blob/use_manylinux1/build-manylinux1.sh

[chezou]

How about pushing linux wheels, which you’ve already released on GitHub, to pypi? I guess his problem is building issue from source code.

[ibayer]

@chezou I think this would lead to trouble since the wheels won't work on all linux distros. :-(

[chezou]

Ah, I missed Pypi requires manylinux whl.

[zouzias]

@ibayer Here is also an example of how to integrate manylinux1 with Travis:

https://github.com/pypa/python-manylinux-demo

Notice that you need to build the wheels for x86_64 and i686.

[chittti]

Here's a script which can build wheels on Ubuntu using docker. The script is pretty straight forward, so you should be able to follow along for different distros.

Soon I will be making a PR which will modify travis.yml so that wheels will be built automatically, so this is just to help someone build them on their own.

https://gist.github.com/chittti/c95de0bd1bab8340c34b59d1be21bb26

which uses the build script for CentOS docker image:

https://gist.github.com/chittti/bfa4f4fdc2d8ab9e8be01ce2dd087cce

I've tested the script in Ubuntu 14.04 and 16.04 so please let me know any of you have problems.

[ibayer]

@chittti Thanks for your contribution. Can you make the gist public and add a license? You could for example add "This project is licensed under the terms of the MIT license." as comment to the script.

[chittti]

@ibayer done. I've licensed it under the same license as this project so it'll be simpler.

[ibayer]

I cut a new release yesterday using our new manylinux setup. :-D

This release adds the capabilities to build binaries (wheels) that work across a large range of Linux distributions to our travis-ci. This will greatly benefit everyone who had issues with source installing fastFM.

Special thanks goes to

@zouzias for pointing us to the manylinux project,
to @chittti for writing the manylinux based build script
and @chezou for integrating the build script in our travis config.