wbolster / plyvel

Plyvel, a fast and feature-rich Python interface to LevelDB
https://plyvel.readthedocs.io/
Other
531 stars 76 forks source link

Add github actions CI/CD for building cross-platform(Linux, MacOS and Windows) wheels for `leveldb 1.23` and `snappy 1.1.9` #129

Open liviaerxin opened 3 years ago

liviaerxin commented 3 years ago

New Features:

  1. Support manylinux_x86_64, manylinux_aarch64, manylinux_i686 for cp37* cp38* cp39* in Linux using cibuildwheel
  2. Support x86_64 for cp37* cp38* cp39* and universal2 for cp39* in MacOS using cibuildwheel
  3. Support win32 and x64 for cp37* cp38* cp39* in Windows using cibuildwheel
  4. Support the version 1.23 of leveldb and the version 1.1.9 of snappy for all platforms(Linux, MacOS, Windows)
  5. Support pytest in CI

Check and verify my new CI result at github actions CI/CD

Released wheels at https://github.com/liviaerxin/plyvel/releases/tag/v1.3.0-CI

TODOs:

  1. Apply a new version such as 1.3.0 for plyvel.
  2. Modify function test_open_read_only_dir() in test/test_plyvel.py to pass pytest.
  3. Publish built wheels to pypi using github action.
wbolster commented 3 years ago

wow, i imagine this was quite some effort. thanks! :pray:

i will have a good look at this later.

AustEcon commented 3 years ago

I did not realize this was here!!

I have been working on this myself on a private repo because I don't want the user to need to be required to do a system install of leveldb..

I have wheels for all platforms here for python 3.6 3.7 3.8 3.9 : https://pypi.org/project/plyvel-wheels/#files

I've posted a PR for what it's worth. Whether you use it or not it's no worries 😄

liviaerxin commented 3 years ago

Wow, @AustEcon thank you for presenting a idea to build windows wheels.

I will try to add it into github actions CI/CD later.

liviaerxin commented 3 years ago

I have add to build windows wheel in github actions CI for leveldb 1.23 and snappy 1.1.9.

Please see new commits and CI results at plyvel github actions CI

odidev commented 2 years ago

@wbolster, when are you planning to review this PR and merge it. It will be a great help to the machine users for which wheels are missing on PyPI.

synodriver commented 2 years ago

I've also made a fork to build wheels for windows which is build against the latest leveldb version(the main branch). Should we consider release the wheels derectly since install leveldb on windows is far more harder than that on unix? I could submit anothor pr for that.

liviaerxin commented 2 years ago

I have released wheels built for Linux, Windows, and macOS. You can download your platform-specified plyvel wheel at https://github.com/liviaerxin/plyvel/releases/tag/v1.3.0-CI and install them directly without building both snappy and leveldb.

wbolster commented 2 years ago

hey @liviaerxin wow, that is fantastic! i see you built on top of my build scripts (which i only ever ran successfully in a local docker environment, not in CI) 👍🏼

it would be great if we can get this cleaned up a bit and make it official, and trigger everything from this repo. we could even do automatic releases + pypi based on git tags etc. are you interested in helping getting that set up?

(i can also arrange write access to this repo of course)

liviaerxin commented 2 years ago

hey @liviaerxin wow, that is fantastic! i see you built on top of my build scripts (which i only ever ran successfully in a local docker environment, not in CI) 👍🏼

it would be great if we can get this cleaned up a bit and make it official, and trigger everything from this repo. we could even do automatic releases + pypi based on git tags etc. are you interested in helping getting that set up?

(i can also arrange write access to this repo of course)

Hello @wbolster, I'm glad to participate in the following work and assist you to publish pypi/release automatically by CI to help people use plyvel package easily.

liviaerxin commented 1 year ago

Hello @wbolster, I have not tracked your plyvel repo for a while and see it still does not provide binary wheels for cross-platform. Now I have some time and really wish your repo to be accessible for more people especially Windows users with you.

I can add some improvement to do automatic releases + pypi based on git tags, but it may need your helps such as setting some secret keys.

If you have other suggestions on CI, such as your mentioned code cleanup, feel free to tell me.

Then I can kick off the pull request again and simplify it by squashing commits.