MarkEWaite
commented
1 year ago I built over 240 of the top 250 plugins with the settings.xml file listed later (using repo.jenkins-ci.org for the releases repository, the jgit repository, and the netbeans repository).
I had to download the jbcrypt library separately but other than that, I was able to build without running tests.
The repositories that did not build with those settings are:
- artifactory-plugin - cannot resolve com.jayway.jsonpath:json-path:jar:2.4.0 from token-macros dependency, works with my default settings (refer to analysis of failure)
- blueocean-plugin - cannot resolve com.atlassian.jira:jira-rest-java-client-api:jar:5.2.1, works with my default settings
- configuration-as-code-plugin - cannot resolve com.atlassian.crowd:crowd-integration-api:jar:5.1.5, works with my default settings (analysis of failure)
- jira-plugin - cannot resolve com.atlassian.jira:jira-rest-java-client-core:jar:5.2.7, works with my default settings (analysis of failure)
- promoted-builds-plugin - http downloads blocked for hudson.plugins:project-inheritance:jar:21.04.03, works with my default settings (analysis of failure)
- robot-plugin - cannot resolve com.jayway.jsonpath:json-path:jar:2.4.0, works with my default settings (analysis of failure)
- scp-plugin (fails to build even with my default settings and fails to build with Java 8 as well)
One plugin (gradle-plugin) does not build with Maven, so I can't check it.
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<pluginGroups>
<pluginGroup>org.jenkins-ci.tools</pluginGroup>
</pluginGroups>
<profiles>
<profile>
<id>jenkins</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<repository>
<id>repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/releases/</url>
</repository>
<repository>
<id>jgit.repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/jgit/</url>
</repository>
<repository>
<id>netbeans.repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/netbeans/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/releases/</url>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
</settings>
timja
daniel-beck
dduportal
basil
jtnord
Service(s)
Artifactory
Summary
The data transfer volume from https://repo.jenkins-ci.org is still unexpectedly high. Log file analysis for a recent 5 day period shows that the top repositories based on the sum of the sizes of the requested files are:
JFrog has explained that even though we removed Apache Maven Central from the public repository, the contents of Apache Maven Central still exist as copies in the jcenter repository. We need to also remove the jcenter repository and the jcenter-cache repository from the definition of the public repository so that binaries from Apache Maven Central will be resolved from Apache Maven Central instead of being resolved from our repository.
When I had tried some initial experiments removing jcenter, several months ago, there were some cases where it appeared that we depend on artifacts that are only in jcenter and possibly only in our cache of jcenter. Unfortunately, my memory is weak and I cannot find any notes that reference that experiment.
I think that we should take the following steps:
com.jayway.jsonpathgroupId and all its packages from thejgit-cache. The artifactory plugin build failure and the robot plugin build failure will be resolved by that exclusion. The comment from @basil includes more detailsweekly-testconfiguration of the plugin bill of materialsweekly-testconfiguration of the plugin bill of materialsReproduction steps
Steps that I took to analyze the requests:
select repo,sum(size_bytes)/1024/1024/1024 as SUM_GB from logs group by repo order by SUM_GB desc limit 10;