search

dcwatson / deflate

Python extension wrapper for libdeflate.
MIT License
25 stars 6 forks source link

new release? #26

Closed ThomasWaldmann closed 1 year ago

ThomasWaldmann commented 2 years ago

is there something left to do for a new pypi release which we can help with @dcwatson?

guess we can't do much about #23 because that is in libdeflate project scope, not in this project's scope (besides maybe doing an update of the subrepo later, IF relevant tuning happened there).

ThomasWaldmann commented 2 years ago

changelog - shall we help? next release is 0.4.0?

ThomasWaldmann commented 2 years ago

29 is a good change and ready to merge into master, should get into next release i guess.

ThomasWaldmann commented 2 years ago

looks like @dcwatson is busy right now, so i'll just work on a changelog PR...

see #30.

ThomasWaldmann commented 2 years ago

btw, @dcwatson, if you like to increase the "bus factor" a bit, you could add me as addtl. maintainer (or owner) for deflate on pypi.org.

my username there is: ThomasWaldmann

i would also offer to do next release in case you don't have time.

ThomasWaldmann commented 2 years ago

29 will need a changelog entry after merge.

dcwatson commented 2 years ago

Biggest blocker (in my estimation) is probably Windows support. Need to make sure this tests/builds wheels on Windows in addition to macOS and Linux. It would also be good to have CI for #29 where we could make sure both paths work, but if that's prohibitive I'm fine doing it manually for now.

ThomasWaldmann commented 2 years ago

when i did some tags recently, it started the wheel build. so guess you could just merge and tag to trigger that.

i'll add a changelog entry now to the PR, so that todo is also done.

ThomasWaldmann commented 2 years ago

as long as pypi pkgconfig is NOT installed in your build action, it should fall back to the bundled code, so same behaviour as before.

ThomasWaldmann commented 2 years ago

found a nice cibuildwheel option to run tests... - trying that in the PR...

ThomasWaldmann commented 2 years ago

can we do a release ASAP? i have 1,2 PRs at borgbackup blocked by this...

dcwatson commented 2 years ago

Sorry for the radio silence, everyone in my house has been sick the past week.

I think this is pretty much ready to go. In thinking about it some more tonight, I think we should break out two packages, deflate and deflate-binary (like what psycopg2 did). Being able to configure how libdeflate is compiled or linked is great, but having pre-built binary wheels is also great for containers and places without compilers. So for 0.4 and forward, I think I will just skip building the wheels for the deflate package, call it the compiled version, and investigate how to make a deflate-binary package happen.

I know you seem to be in a hurry for a new release. Do you have any problem with a 0.4.0 release without binary wheels, until I can figure out a script for building a deflate-binary package from this repo?

ThomasWaldmann commented 2 years ago

Oh, i wish you all a quick recovery!

Timing is relaxed now, because I solved the issue in another way:

In borg, we already do a lot of interfacing to C libraries via Cython (e.g. to lz4, zstd, xxh64), so I decided to just do it in the same way for libdeflate. It's more consistent like that, was pretty easy (as it is only the one wrapper for crc32) and avoids adding a python dependency for deflate.

deflate python pkg was very useful to us for evaluating libdeflate performance on misc. platforms. On macOS, we'll use zlib.crc32 now, on Linux and all other platforms we'll use libdeflate_crc32.

ThomasWaldmann commented 2 years ago

About the binary package / wheels: I don't have strong opinions here.

william-silversmith commented 1 year ago

Hi all! Would it be possible to get a new release of py310 and py311 wheels? Not super urgent, but would be good! Just need to bump the cibuildwheel version for GHA.

I know it's been a while, but I hope you're feeling better dcwatson!

dcwatson commented 1 year ago

Finally pushed out 0.4.0 to PyPI, including pre-built wheels for Python 3.8-3.11 on all platforms. I still think it's worth splitting into two separate packages (source vs. binary) but it's been long enough...