More robust artifacts publishing

[iNikem]

We have experienced a lot of problems recently during artifacts upload and publishing. This task tries to summarise my understanding of better options to achieve the desired outcome.

First, the least desirable option is to abandon Bintray/JCenter altogether and publish directly to MavenCentral. I don't have first-hand experience with this, but Internetz have scared me away from that option. I am under impression that publishing directly to Maven Central is not for the faint of heart. Thus below I assume that we want to continue to distribute our artifacts to MavenCentral via JCenter.

One option is stop using bintray gradle plugin and to use standard gradle's maven-publish plugin to upload artifacts to Bintray. That should work without problems but will require two "manual" steps: creating to-be-released version in Bintray and then actually publishing uploaded artifacts to that version, thus making them visible to everybody. Both steps can be achieved by Bintray REST API.

The second option is to use Artifactory for snapshot publishing and then using Artifactory promotion feature to push already built artifacts to Bintray. Uploading files to Artifactory works without problems for us. And this will eliminate the need to build and test artifacts once more before the release. We just move existing artifacts "snapshot" repo to Bintray.

This obviously requires some Artifactory instance :) There is Cloud Artifactory, but free tier will certainly not satisfy us. Thus we have to find some sponsor to pay for it. Some options are:

• CNCF will pay for it and will try to make a good deal with JFrog.
• Some vendor (Splunk, Microsoft, etc) can sponsor us.
• We can use already existing instance of Artifactory used by some vendor. E.g. Splunk already has one in house.
• We can apply for OSS sponsorship from JFrog directly.

[anuraaga]

Thus below I assume that we want to continue to distribute our artifacts to MavenCentral via JCenter.

This seems quite early to assume in the discussion ;)

From what I understand there is no such thing as publish straight to Maven Central - you publish to a different Maven repo and sync to it. For OSS, JCenter is the Artifactory instance to do that, oss.sonatype is the Nexus instance to do that.

The main reason I've seen for people to go with JCenter is not understanding crypto. It takes care of GPG signing - this is convenient since dealing with keys is scary, but really we have so many keys so it's probably not that bad pointing Gradle at a key IMO. And arguably we should be doing it anyways with a CNCF key.

So that doesn't mean jfrog isn't a good option but I don't think we can rule others out. My only anecdote is Zipkin has had much trouble with bintray. We've migrated everything to oss.sonatype and had one successful release across many repos. Only one so far. It's only stable until it all breaks down so I can't give any resounding endorsement but let's not ignore options. The only thing I can say from my own experience is I've seem lots of badness from jfrog (edit: I'm using jfrog to refer to both bintray and artifactory snapshots).

[anuraaga]

By the way, just to clarify Cloud Artifactory may still be a good option for us, but just that it'd be good to check out all the options :) I'm wondering what we get from proxying through to sonatype oss and then to maven central, aside from easy GPG - I've seen failed syncs from bintray where we'd need to login to sonatype oss to delete the staging repo there anyways, so I guess it's a dependency regardless of where we push to before, maybe can reduce some running parts.

[trask]

I looked around but didn't find any mention of Cloud Artifactory having a feature to sync to maven central. I wonder if syncing to maven central is only a bintray thing? #1778 seems like our last shot with bintray (in case signing locally reduces bintray publishing load), and if that doesn't work, I'm not sure we have any other option besides sonatype?

[iNikem]

Artifactory can publish to Bintray which then can sync to Maven Central

[iNikem]

One potential benefit of using Artifactory is the ability to attach build info to all published artifacts. E.g. attaching git sha to them to be able to backtrack any jar to the commit it was built from. And any other metadata we may wish.

[iNikem]

But for the record: I am OK with any solution that works and is stable. Just if I am going to do it, I will first try to go Artifactory way :)

[anuraaga]

I'd like to give sonatype a try - the build configuration change is pretty small so if it doesn't work out it's easy to back out

https://github.com/aws/aws-xray-sdk-java/blob/master/build.gradle.kts#L279

We need to get the sonatype user / pass into this repository, which we needed eventually for Maven Central anyways so now's a good time to do that. I guess only @carlosalberto has that right now? We should probably also get all the Java maintainers sonatype access to be able to debug issues with maven central sync (think we need it regardless of our publishing proxy anyways)

[iNikem]

One word of caution: we will create confusion among our users who are using bintray right now and all of a sudden has to switch repo for newer releases

[anuraaga]

@iNikem It's a good point. @trask Are we ready to publish to Maven Central (renaming is done)? If so, we can do the switch now

[trask]

I'll review artifact names today.

[trask]

Closing, 0.12.0 is published to maven central, and Sonatype publishing has been more reliable so far at least (hopefully not just beginners luck 🍀).