Automate the release

[pjfanning]

• sbt has good plugins for doing releases to Nexus - note ASF has its own Nexus instance, we don't use Sonatype. https://repository.apache.org/
• also need to gpg sign and generate SHA256/SHA512 digests for the source and binary distributions (#78)

[mdedetrich]

Since the core pekko build is really complicated, we could use https://github.com/apache/incubator-pekko/issues/92 as a test bed for creating an automated release (its going to be needed at some point anyways). Once its figured out and working for sbt-paradox-pekko its going to be a lot simpler to then just add it into pekko.

In regards to the plugins to automate a release, the most ideal would be to use sbt-ci-release however it does seem to be tied to sonatype, the nice thing about sbt-ci-release is it completely automates the process, a release is triggered just be pushing a git tag but this may be too modern for Apache and might go against some of their established processes. Another option is sbt-release which is a bit more manual/old-school however it allows you to customize the release process as you want and then you would just do an sbt release to trigger a release. Also as a bonus, @jrudolph has actually contributed to sbt-release so we have some familiarity with it.

[pjfanning]

I use sbt-typelevel-ci-release - which appears to be a more active copy of sbt-ci-release.

[mdedetrich]

Afaik its not that sbt-typelevel-ci-release is more maintained but rather it contains additional typelevel'isms related to their org's standards for releasing software.

In any case I was talking casually about these sbt release plugins to other Apache people and there appeared to be some hesitation, one given example is whether its proper in the eyes of Apache Security to store keys as secrets on github (which would be necessary for sbt-ci-reease/sbt-typelevel-ci-release). On the surface sbt-release seems to be closer to the Apache way of releasing things and most importantly isn't tied to sonatype.

As an example, since the steps in sbt-release can be completely customized, its possible to put the "copy source code into Apache's SVN repo" as part of the release step (at least in Apache's view, this is the only thing that constitutes as an actual release. Everything additional such as uploading jar's to maven is just for user convenience).

[mdedetrich]

This PR is related to using sbt-ci-release to do release management https://github.com/apache/incubator-pekko/pull/129 . In summary it may be possible to use sbt-ci-release by just changing the sonatype host to repository.apache.org, hopefully the Apache Nexus repo is close enough to OS sonatype that they accept the same way of publishing (i.e. bundle releases may not work with Apache Nexus, we will have to find out).

If not then we will have to use something like sbt-release which isn't hardcoded to use sbt-sonatype.

[jrudolph]

• we don't use Sonatype

Maybe we should clarify what a release is and what the desired outcome of doing a release is. How do artifacts end up on Maven Central if not going through Sonatype? Let's start listing all the steps necessary to do different kinds of releases (snapshots, RCs, and GA) and only then start about automating any of it. I imagine this is at least partly about what Apache considers a release? If yes, then we should clarify what it entails.

I would consider sbt-ci-release to be a tool to work with sonatype. If we don't go through sonatype (or a process very similar to sonatype with staging repos etc), we probably should not use it. Afaik, the main reason it was created was that sonatype (Nexus) needs some manual handholding to finally publish something to Maven Central.

AFAIK sbt-release is a plugin to organize a release-process that has multiple steps but is more abstract than publishing to a repository.

If this is just about pushing artifacts to a maven-compatible repository, we might not need any plugins for just that.

[mdedetrich]

One of the nice things about using sbt-release instead of sbt-ci-release is that as you just pointed out, you can customize it to do whatever you want. For example even though we publish generated library jars to Apache's Nexus repo, this is not considered an official. An official release involves copying the sources over using rsync, so we can theoritically use sbt-publish-rsync or fork it for Apache's needs, and then we can can completely automate publishing both to Nexus and making an official Apache release.

For now I am just trying to get snapshots working, so I don't see a problem to see if we can adjust sbt-ci-release to work with Apache's Nexus repo however I am still personally leaning to using sbt-release for reasons stated earlier.

[jrudolph]

I created https://github.com/apache/incubator-pekko/issues/130 to try to collect an overview over all the steps that are needed. This ticket would be a subticket of the complete process.

[pjfanning]

@jrudolph repository.apache.org artifacts get synched to Maven Central - but I think only when you do full releases. Snapshots and staged (pre-release) artifacts are probably only accessible by adding an sbt resolver that has the repository.apache.org url set up.

[mdedetrich]

@jrudolph repository.apache.org artifacts get synched to Maven Central - but I think only when you do full releases. Snapshots and staged (pre-release) artifacts are probably only accessible by adding an sbt resolver that has the repository.apache.org url set up.

True but its also nothing special, at least for sbt even the OS Sonatype snapshot repository needs to be manually added as a resolver (i.e. its not there by default).

[jrudolph]

Yes, and that's totally fine and also pretty much wanted (also according to https://infra.apache.org/release-distribution.html#unreleased which explicitly require "must not be distributed through channels which encourage use by anyone outside the project development community").