Discussing the release pipeline

[amotl]

Introduction

We are coming from #341 and #423 here.

After #423 has been merged (thanks again @koenvervloesem and @jpmens), we might consider building a full release pipeline in order to get rid of any tedious work involved when cutting a new release. While I generally hesitate to over-engineer things, we might come to the decision that it could be worth putting in some effort once in order to save us from any future grunt work on that topic.

Coordinating releases

Currently, cutting releases and publishing the release artifacts involves all of the core maintainers.

Example 1

@jpmens said:

I published using an automated build which is set up to "when push to mqttwarn, build".

@rgitzel answered:

Rather than republishing the image automatically on a Git commit, it is probably clearer to do the build/tag/push steps manually when a new version of the app is released. Probably the Dockerhub automation could do that, but that might be more complicated to set up than it's worth?

Example 2

@amotl asked:

Shall I cut a new release 0.16.0 collecting all the latest updates and publish this to PyPI before you will build and publish a new image to Docker Hub?

@jpmens answered:

Go ahead, @amotl.

State of the onion

I recognize that some bits and pieces already got automated in some way:

• @amotl added a "make release" target to automate cutting a release and publishing a sdist-package to PyPI the other day. https://github.com/jpmens/mqttwarn/blob/0.15.0/Makefile#L49-L52
• @amotl and @jpmens established running the tests on CircleCI. https://app.circleci.com/pipelines/github/jpmens/mqttwarn
• @jpmens uses docker cloud to build and publish Docker images.

Thoughts

Is anyone interested in helping out with this? I see these topics worthwhile to include in the respective discussion:

• Automate the basic release process: Bump version using bump2version, Create Git tag, Push to repository, Create sdist package, Upload to PyPI. This is already covered by make release conveniently and - in my opinion - should be done manually as it is the obligation of the developer/maintainer.

• Invoke the Docker image building: Besides the regular image jpmens/mqttwarn, I would like to see a mqttwarn-full image like what @rgitzel started already.

• Also, we might integrate what @koenvervloesem is suggesting within #23 and #424 somehow.

With kind regards, Andreas.

[amotl]

I had the pleasure to work together with @gutzbenj on the very same aspects within the fine Wetterdienst package, see also https://github.com/earthobservations/wetterdienst/issues/85. @gutzbenj just added a corresponding GitHub Action release.yml we might want to consider reusing here.

Please note that we already switched from setup.py to pyproject.toml and Poetry there, see https://github.com/earthobservations/wetterdienst/pull/90.

[amotl]

Hi again,

I believe the release pipeline now is in a decent shape, so I am closing this ticket.

With kind regards, Andreas.