search

dask / hdfs3

A wrapper for libhdfs3 to interact with HDFS from Python
http://hdfs3.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
136 stars 41 forks source link

Publish wheel to PyPI that includes binary dependencies #113

Open nchammas opened 7 years ago

nchammas commented 7 years ago

Following up on #64, I'm wondering if there is a way to update our packaging so that users can simply pip install hdfs3 without needing to independently install libhdfs3 using their system package manager. Currently, the pip installation docs include a separate apt-get step, which can definitely be pain depending on your OS and distribution.

There is some information on the Python Packaging Guide about publishing binary extensions, which I think is what we need here, but the docs are incomplete. I also know of a project that bundles a C extension in its package on PyPI, but I don't know if that example is relevant to what we would need to do here.

dstufft commented 7 years ago

You probably want https://github.com/pypa/manylinux and https://mail.python.org/mailman/listinfo/wheel-builders

mrocklin commented 7 years ago

I entirely support this effort and am happy to support in any way (such as by giving push access to PyPI). However I'm also unlikely to take a lead on this.