Remove `jcenter`, and `oss.sonatype.org-releases` repositories from `public` virtual repository; reconfigure Atlassian remote repositories

[MarkEWaite]

Service(s)

Artifactory

Summary

The data transfer volume from https://repo.jenkins-ci.org is still unexpectedly high. Log file analysis for a recent 5 day period shows that the top repositories based on the sum of the sizes of the requested files are:

Repository	Sum of GB requested
releases	2380
jcenter-cache	1787
incrementals	298
jcenter	98
nodejs-dist-cache	78
public	26
jgit-cache	14
oss.sonatype.org-releases-cache	8
snapshots	8
npm-dist-cache	5

JFrog has explained that even though we removed Apache Maven Central from the public repository, the contents of Apache Maven Central still exist as copies in the jcenter repository. We need to also remove the jcenter repository and the jcenter-cache repository from the definition of the public repository so that binaries from Apache Maven Central will be resolved from Apache Maven Central instead of being resolved from our repository.

When I had tried some initial experiments removing jcenter, several months ago, there were some cases where it appeared that we depend on artifacts that are only in jcenter and possibly only in our cache of jcenter. Unfortunately, my memory is weak and I cannot find any notes that reference that experiment.

I think that we should take the following steps:

• [x] Create a new repository on repo.jenkins-ci.org and copy the org.connectbot.jbcrypt:jbcrypt:1.0.0 artifact from jcenter-cache to the new repository (required for Jenkins core, appears to only be available from jcenter-cache). Add the new repository to the public virtual repository
• [x] Announce a 2 hour brownout where jcenter and jcenter-cache will be temporarily removed from the public repository
• [x] Exclude the com.jayway.jsonpath groupId and all its packages from the jgit-cache. The artifactory plugin build failure and the robot plugin build failure will be resolved by that exclusion. The comment from @basil includes more details
• [x] Add orphans repository to the public virtual repository
• [x] Create a new remote repository corresponding to https://packages.atlassian.com/maven-public/ and replace the existing atlassian entry with the new one
• [x] Define the tests we want to perform during the 2 hour brownout, including:
  • Build Jenkins core on ci.jenkins.io
  • Build Jenkins core on a freshly created virtual machine outside any CI infrastructure
  • Build 20+ popular plugins on ci.jenkins.io
  • Build 20+ popular plugins on a freshly created virtual machine outside any CI infrastructure
  • Build plugins that have shown issues (click links for analysis) including artifactory-plugin, blueocean-plugin, configuration-as-code-plugin, git-client-plugin, jira-plugin, promoted-builds-plugin, robot-plugin
  • Build plugins that require Java 8 and have shown issues including pipeline-aws-plugin, translation-plugin
  • Run the ci.jenkins.io weekly-test configuration of the plugin bill of materials
  • Build key Jenkins tools like the Maven HPI plugin and the plugin compatibility tester
• [x] Prepare machines to test the brownout
  • [x] Jenkins core build machine with Java 21 and Apache Maven 3.9.6 installed. Clone the Jenkins core repository to the machine but do not compile until the brownout has started
  • [x] 5 Jenkins plugin build machines with Java 8, Java 11, Java 17, Java 21 and Apache Maven 3.9.6 installed. Clone the specific plugin repositories but do not compile until the brownout has started
• [x] Run the brownout and perform the tests, collecting the results
  • [x] Build Jenkins core on ci.jenkins.io
  • [x] Build Jenkins core on a freshly created virtual machine outside any CI infrastructure
  • [x] Build 20+ popular plugins on ci.jenkins.io
  • [x] Build 20+ popular plugins on a freshly created virtual machine outside any CI infrastructure
  • [x] Build plugins that have shown issues (click links for analysis) including artifactory-plugin, blueocean-plugin, configuration-as-code-plugin, git-client-plugin, jira-plugin, promoted-builds-plugin, robot-plugin
  • [x] Build plugins that require Java 8 and have shown issues including pipeline-aws-plugin, translation-plugin
  • [x] Run the ci.jenkins.io weekly-test configuration of the plugin bill of materials
  • Identify failures and investigate alternatives for those failures (for example, if we discover a library that is only available in jcenter, we can copy that library to another Jenkins repository)
• [x] Reset the public configuration at the end of the brownout
• [ ] Implement the configuration change
• [ ] Request and analyze log files for the 5 day period after the configuration change
  • Confirm the jcenter-cache and jcenter repositories are no longer in the top 5 repositories

Reproduction steps

Steps that I took to analyze the requests:

1. Request Artifactory log files from JFrog
2. Update my artfiactory-sql fork to handle the new format of Artifactory log files
3. Upload the log files to a sqlite3 database
4. Identity top repositories with the SQL query select repo,sum(size_bytes)/1024/1024/1024 as SUM_GB from logs group by repo order by SUM_GB desc limit 10;

[dduportal]

• Removed jcenter-cache and atlassian-cache from both Any Remote and Anything permissions scheme to ensure they cannot be reached anonymously (as such the outbound bandwidth should drastically decrease)

Thanks very much. Should the same change be applied to oss.sonatype.org-releases since it also contains a copy of Apache Maven Central?

The repository oss.sonatype.org-releases is no longer available anonymously: it requires authentication.

[dduportal]

Update:

• The repository orphans has been removed (in favor of adding filters in the jcenter-orphans mirror repository - https://github.com/jenkins-infra/helpdesk/issues/3884
• We forgot to clean the ACP cache: https://github.com/jenkins-infra/helpdesk/issues/3889