Ability to provide a jenkins URL for plugins

[atyrode]

Enhancement Type

Improve an existing feature

Describe the enhancement

Referencing this discussion: https://github.com/itzg/docker-minecraft-server/discussions/2273#discussioncomment-7219702

Some plugins distribute their JARs through Jenkins or other CI/CD platforms, distributed at the /lastSuccesfulBuild/artifact endpoint in the case of Jenkins:

https://ci.citizensnpcs.co/job/Citizens2/lastSuccessfulBuild/artifact/

Unfortunately, sometime there's multiple artifacts in the build:

https://ci.athion.net/job/FastAsyncWorldEdit/lastSuccessfulBuild/artifact/artifacts/

Even if there's only one artifact, you can't resolve the download link without giving the build number unless the project set a fixed name for the artifact because Jenkins doesn't provide a latest build endpoint that's version agnostic.

---

Manually resolving:

1. /lastSuccesfulBuild/api/json contains a filename key which points to the artifact
2. /lastSuccessfulBuild/artifact/*zip* provides a zipped archive with the following structure

   archive/foo/bar/plugin.jar foo/bar being the folder structure of the build

---

Problems:

Some plugins don't provide a clear latest endpoint as well, and uses their own CI/CD:

https://apps.methodscript.com/builds/commandhelperjar/

---

My workaround:

I've added some bash logic that executes before the entry point, which is a bit lengthy because jq is not present in the base image.

    entrypoint: >-
      /bin/sh -c "
      mkdir -p /data/plugins;
      wget -q -O /data/plugins/commandhelper-3.3.5-SNAPSHOT-full.jar --backups=0 "$(curl -s "https://apps.methodscript.com/builds/commandhelperjar/" | grep -oE 'build-[0-9]+' | sort -V | tail -n 1 | awk -F '-' '{print "https://apps.methodscript.com/builds/commandhelperjar/build-"$2"%2Fcommandhelper-3.3.5-SNAPSHOT-full.jar"}')";
      wget -q -P /data/plugins --backups=0 'https://ci.athion.net/job/FastAsyncWorldEdit/lastSuccessfulBuild/artifact/artifacts/'$(curl -s https://ci.athion.net/job/FastAsyncWorldEdit/lastSuccessfulBuild/api/json | grep -o '\"fileName\":\"[^\"]*jar\"' | head -n 1 | sed -e 's/\"fileName\":\"//' -e 's/\"$//');
      wget -q -P /data/plugins --backups=0 'https://ci.citizensnpcs.co/job/Citizens2/lastSuccessfulBuild/artifact/dist/target/'$(curl -s https://ci.citizensnpcs.co/job/Citizens2/lastSuccessfulBuild/api/json | grep -o '\"fileName\":\"[^\"]*jar\"' | sed -e 's/\"fileName\":\"//' -e 's/\"$//');
      /start"

_note: that the code above will make SPIGETRESSOURCE fail, most likely due to creating the plugins folder manually

---

Enhancement ideas:

1. (probably unsafe) Passing a bash sequence to mc-image-helper, which it'll evaluate in the context of computing a string, which it will expect to be a download link
2. (Jenkins only I think) Passing a sort of key/value format string where key is the Jenkins link (https://ci.citizensnpcs.co/job/Citizens2) and value being glob-style, like Citizens*.jar and it'll match to a file in the zip.
3. Re-using the same idea as above, perhaps providing directly the CI/CD json endpoint, and the glob formatted value, and it'll try to compute the download path

---

Thanks for considering this enhancement!

[itzg]

Thanks for writing this up.

So the FastAsyncWorldEdit is an interesting case since there's two files per build: FastAsyncWorldEdit-Bukkit-2.8.1-SNAPSHOT-575.jar vs FastAsyncWorldEdit-CLI-2.8.1-SNAPSHOT-575.jar. I'm thinking the reference to each "jenkins URL" will need to also indicate some kind of filename substring to match, which would be "Bukkit" in this case. Does that seem like what you'd expect?

[atyrode]

@itzg I was in the process of reformatting the issue and came up with a similar solution: I'm thinking that Jenkins can be resolved using only a key/value pair, yes.

However, for other implementations such as CommandHelper, beside providing a bash sequence that computes the string, I'm not seeing any robust solutions.

*actually, an alternative would be to provide an api/json endpoint, a link, and a key path if link is empty, it crawls along the key path in the json, and assume that the last value is the endpoint otherwise, it appends it to the link this would cover a broader amount of CI/CD platforms

[itzg]

CommandHelper seems to be an especially strange and difficult one to automate generic retrieval. It's also strange since they used to provide github releases (https://github.com/EngineHub/CommandHelper/releases), but seem to have stopped with 3.3.5.

For Jenkins and Github I don't mind adding highly specific logic to retrieve from those -- do you happen to know what build system CommandHelper is using? I also wouldn't mind specific logic to process the build manifest json you linked, but wouldn't know what to refer to it as.

[atyrode]

Regarding their build system, I think Azure DevOps, based on their architecture documentation:

The manifest json it creates is very annoying to work with, because it does not tell you the latest build. You have to assume it based on the build number, which is very wonky.

---

Also; while trying to go with the approach of storing the links directly in the PLUGINS variable as a fix to my initial workaround issue, I encountered the following issue:

me.itzg.helpers.http.FailedRequestException: HTTP request of
https://apps.methodscript.com/builds/commandhelperjar/build-364%2Fcommandhelper-3.3.5-SNAPSHOT-full.jar
failed with 405 Method Not Allowed: Extracting filename

I'm not well versed in URL encoding but they're not distributing the file to:

.../build-364/commandhelper-3.3.5-SNAPSHOT-full.jar