[JENKINS-28502] fixed Groovy crawler

[daspilker]

Groovy distributions are hosted on Bintray after Codehaus was shut down.

In line 15 I had to remove a leading # from the URL which is done by JavaScript on the Bintray page.

[jenkinsadmin]

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

[KostyaSha]

Bintray has API, it will be more correct just query their API, but it requires some registered user.

[daspilker]

An API key is needed which could be provided by an environment variable, but that requires changes outside of this repo.

I would like to keep the change simple to get things fixed. How about creating an INFRA issue for using the Bintray API, so that it can be changed later on?

[KostyaSha]

UI name that mentions "codehaus" need to be fixed also.

[daspilker]

Good point, but that needs to be fixed in the Groovy plugin, see https://github.com/jenkinsci/groovy-plugin/pull/16.

[daspilker]

Are there any issues left with this PR? Can it get merged?

[orrc]

As mentioned in JIRA, the older Groovy releases aren't available at https://dl.bintray.com/groovy/maven/

So merging this will break any Jenkins jobs configured that depend on Groovy versions older than 2.2.0 (i.e. virtually all of them).

The output of this PR needs to match (or include at least the contents of) the existing output: https://updates.jenkins-ci.org/updates/hudson.plugins.groovy.GroovyInstaller.json

[daspilker]

No, it's currently broken anyway since the releases are no longer available at the old URLs. Try any of the URLs in the JSON. So currently it's not possible to install any version of Groovy.

With this pull request, it will at least be possible to use the latest versions. And if the Groovy team pushed the old releases to Bintray, they will automatically be picked up.

[daspilker]

Ah, I got what you mean. If Groovy is already installed on a node, this will break the job config. But when creating new node (e.g. by any of the cloud providers) with no Groovy installed, there is currently no way to install Groovy with the auto-installer.

[KostyaSha]

@daspilker either if you kill tools cache.

[daspilker]

I hard coded the old Groovy versions to the script in an extra commit, so that the commit can be reverted when the older versions are available on Bintray. So the generated JSON will contain the old hard coded versions and the new versions from Bintray.

[KostyaSha]

They should be available in bintray. Have you checked?

[daspilker]

Yes, not available yet, see https://dl.bintray.com/groovy/maven/. There was a thread on the Groovy dev list, but there seems to be no progress.

[aheritier]

The groovy team (cc @glaforge @melix) retrieved a copy of all binaries but they now need to upload this huge set of data on bintray (cc @freddy33). I don't know if we can have an ETA but this is critical for many Jenkins users relying on its ability to deploy any groovy version (recent or not).

[freddy33]

We just finished the import into Bintray, thanks to @DarthFennec. All versions that were in codehaus should be there now. Enjoy!

[aheritier]

Thanks a lot the #JFrog team @freddy33 & @DarthFennec ! Let's make #jenkinsci and #groovy users happy now (cc @daniel-beck )

[daspilker]

OK, great. I removed the workaround for the old releases.

[daniel-beck]

Thanks @freddy33 and @DarthFennec!