Automate Rust release process

[0xJepsen]

Description

Release-plz has been really nice to use and automates semantic versioning and release tags / even auto pushes to crates.io. I am happy to draft an example PR for. I think it would improve stability and help downstream projects deal with updating polar versions safely.

Link

https://release-plz.ieni.dev/

[MarcoIeni]

Hi, I'm the release-plz author, I found this issue randomly (thanks Waylon for you kind words ❤️). I never used polars, but I heard many podcasts about it, and I love what you folks are doing. 🙌 If you need any help from me, let me know. 👍

[stinodego]

I'm pretty happy with our Python release process. The Rust process is currently manual and should be automated.

I'm not really familiar with release-plz, but from what I have seen, it wouldn't work well together with our current release flow / changelog generation.

[MarcoIeni]

I'm happy to help you automate your rust release flow, either using release-plz or another tool 👍 What does your current manual rust release flow look like? How does it integrate in your python release flow?

Btw release-plz is pretty flexible, you can disable changelog generation, git tag and git release if another tool already does it for you. For example, you could use it just to:

• bump versions in Cargo.toml in a github pr. In this pr you will also get a report of semver check compatibility, so that you don't break your users cargo build
• run cargo publish of all your crates in order when you merge the pr

[stinodego]

So far I've had trouble convincing @ritchie46 that automating the Rust release process is a good idea 😅

We use release-drafter to generate the changelog in the GitHub (separate changelogs for Python / Rust). And there's a manually triggered release workflow. Bumping the version is manual at the moment. You can check out the release workflow here: https://github.com/pola-rs/polars/blob/main/.github/workflows/release-python.yml

For the Polars CLI I have written workflows to bump the version / do the release too (includes crates.io). I'd imagine something similar for this repo: https://github.com/pola-rs/polars-cli

But yeah, I am mostly waiting for Ritchie to give the go-ahead for the Rust side. But I'd be interested to hear what the benefit would be to use release-plz for a workflow like this.

Most important part of the release process is that all artifacts are built and ready to ship before we push to PyPI / crates.io and make a GitHub release.

[MarcoIeni]

Sorry for the late reply. With release-plz you can automate Rust releases but not python ones. If I had time, it would be cool to add support for other languages such as python and javascript, but I don't have the capacity for now.

The main benefits of using release-plz in this scenario I see are:

• you don't need to bump the crate versions manually, release-plz does it for you based on conventional commits. If an API breaking change is detected (with cargo-semver-checks) release-plz bumps the version accordingly, so you don't risk to break your users
• release-plz analyzes which commit impacts every crate, and maintains a CHANGELOG per rust crate, if this is something that can be useful to polars users
• release-plz runs cargo publish in the right crate order. For polars-cli it's easy because it's only one package. But in this repo, you need to calculate the dependency graph and determine the right order.
• For more info you can read the docs :)

Most important part of the release process is that all artifacts are built and ready to ship before we push to PyPI / crates.io and make a GitHub release.

For this, I tried the following workflow:

1. release-plz creates the github release as draft
2. the github action is triggered when the draft pr is created
3. update the binary artifacts to the release
4. remove the draft state from the release

Unfortunately point 2 is not possible in GitHub. So in order to do this:

• disable release-plz github releases and create the github release yourself
• implement https://github.com/MarcoIeni/release-plz-action/issues/88 and run steps 3 and 4 in the same workflow as release-plz, if a release was created.