search

stephan-hof / pyrocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
150 stars 169 forks source link

Push on PyPI #29

Closed sylvinus closed 9 years ago

sylvinus commented 9 years ago

Would be nice to see your package there!

Cheers,

prologic commented 9 years ago

:+1: Let's also get a decent README.rst going as well as nicely packaged wheels for various platforms!

Downchuck commented 9 years ago

That rocksdb shared library is rather large, would you expect the wheel to include that dependency?

prologic commented 9 years ago

Yes I would for various common platforms :)

I'd probably built it for Debian/Ubuntu/CentOS

cheers James

James Mills / prologic

E: prologic@shortcircuit.net.au W: prologic.shortcircuit.net.au

On Sat, Aug 22, 2015 at 8:23 AM, Charles Pritchard <notifications@github.com

wrote:

That rocksdb shared library is rather large, would you expect the wheel to include that dependency?

— Reply to this email directly or view it on GitHub https://github.com/stephan-hof/pyrocksdb/issues/29#issuecomment-133579186 .

Downchuck commented 9 years ago

From wheels: "PyPI currently only allows uploading platform-specific wheels for Windows and Mac OS X".

From the linux side, I had no issue simply dropping in the ".so" file as built elsewhere. Would be nice if the rocks team released a downloadable .so. But that's a different issue.

prologic commented 9 years ago

Really? I've seen LInux wheels up on PyPi and it seems to work just fine? wierd :P

James Mills / prologic

E: prologic@shortcircuit.net.au W: prologic.shortcircuit.net.au

On Sat, Aug 22, 2015 at 10:23 AM, Charles Pritchard < notifications@github.com> wrote:

From wheels: "PyPI currently only allows uploading platform-specific wheels for Windows and Mac OS X".

From the linux side, I had no issue simply dropping in the ".so" file as built elsewhere. Would be nice if the rocks team released a downloadable .so. But that's a different issue.

— Reply to this email directly or view it on GitHub https://github.com/stephan-hof/pyrocksdb/issues/29#issuecomment-133595747 .

stephan-hof commented 9 years ago

Hey,

I will have a look for uploading it on PyPi. I think uploading it as a 'source' distribution is fairly easy and possible.

However as Downchuck says, Linux wheels can not be uploaded there. Armin R. wrote about it here: http://lucumr.pocoo.org/2014/1/27/python-on-wheels/

For instance Linux binary distributions are still not an option for most people because of two basic problems: Python itself being compiled in different forms on Linux and modules being linked against different system libraries. The first problem is caused by Python 2 coming in two flavours that are both incompatible to each other: UCS2 Pythons and UCS4 Pythons. Depending on which mode Python is compiled with the ABI looks different. Presently the wheel format (from what I can tell) does not annotate for which Python unicode mode a library is linked. A separate problem is that Linux distributions are less compatible to each other as you would wish and concerns have been brought up that wheels compiled on one distribution will not work on others.

The thing what comes closest to a official statement is this: http://python-packaging-user-guide.readthedocs.org/en/latest/current.html#id8

PyPI currently only allows uploading Windows and Mac OS X wheels, and they should be compatible with the binary installers provided for download from python.org. Enhancements will have to be made to the wheel compatibility tagging scheme before linux wheels will be allowed.

Semi official the answer of Nick Coghlan here https://twitter.com/mitsuhiko/status/426700148409135104

@prologic could you post which packages you have seen, having Linux wheels on PyPi ? Keep in in mind, pyrocksdb ships a C-Extensions => A wheel for it must be platform specific, because its compiled against a certain python version. So the example you may post must also have C-Extension.

I have bad news regarding other platforms like Mac/Windows. I'm a Linux developer/user => I don't have access to Mac/Windows boxes, so it will be hard for me to generate wheels for this platforms. Here I need help from somebody else. Any developer here who already tried rocksdb for windows https://github.com/facebook/rocksdb/pull/646 ?

What I could image for a Mac/Windows wheel is a 'static' build against rocksdb. So there is no need to install/build rocksdb on the target machine. Just a pip install pyrocksdb does the trick. I know _pyrocksdb.so will be huge, because it has all rocksdb code in it, but who cares ?! Regarding this 'static build' against rocksdb I will do some experiments on Linux writing my findings here.

prologic commented 9 years ago

I think I source dist up on PyPi is fine :)

stephan-hof commented 9 years ago

Here we are: https://pypi.python.org/pypi/pyrocksdb

If there are mac/windows developers out there who would like to support me in generating a 'binary' wheel. Let me know. I played around with the static build and it seems to work. So on this platforms the wheel could already include the compiled rocksdb.

prologic commented 9 years ago

Nice one :+1: