Open shadycuz opened 2 years ago
As we discussed, https://github.com/jenkinsci/custom-war-packager is a way to do it. To be updated
I think the custom-war-packager
is a great project and makes a lot of sense to use for people wanting to manage their docker container while getting benefits from tools like dependabot
.
But I don't think it is the solution for this issue, which is how do I make it easier for non java folks to use JFR. I do think providing a dockerfile
using one or both templates I outlined above would close this issue.
Last time I tried to do this I failed, but It's been a year and I know a lot more about java, so maybe I will find time to try again.
I agree @shadycuz . Note that both issues for Tekton and GitHub actions target such integrated experience. I think the same should be done for the Docker images, we can indeed integrate the Custom WAR Packager mode there right away. I will add it as a separate item to the roadmap if you do not mind
FTR Issues
@oleg-nenashev So I spent a couple hours today trying to tackle this issue. I made progress but eventually got stuck.
Here is the Dockerfile
I am trying to make work;
ARG baseImage=jenkins/jenkins:lts
FROM jenkins/jenkinsfile-runner:latest as jfr
FROM ${baseImage}
COPY --from=jfr /app /app
ENV JENKINS_HOME="/usr/share/jenkins/ref/"
ENV JAVA_OPTS="-Djenkins.model.Jenkins.slaveAgentPort=50000 -Djenkins.model.Jenkins.slaveAgentPortEnforce=true -Dhudson.model.LoadStatistics.clock=1000"
ENTRYPOINT ["/app/bin/jenkinsfile-runner",\
"-w", "/usr/share/jenkins/"]
CMD ["-f /workspace/Jenkinsfile"]
At first I tried just copying over /app/bin/jenkinsfile-runner
but that binary doesn't seem to be complete with out the rest of the /app
contents.
Once I had that figured out I built the image and tagged it as jsl_jfr
:
docker build -t jsl_jfr -f docker/jfr/Dockerfile .
When I ran it I got a very strange error:
shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile jsl_jfr
no Jenkinsfile in current directory.
This is strange because the Jenkinsfile is present:
shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile --entrypoint /bin/bash -it jsl_jfr
root@c8ef72baba98:/# cat /workspace/Jenkinsfile
node() {
println('working')
}
root@c8ef72baba98:/#
Then I tried to run it manually and things got weirder:
root@c8ef72baba98:/# /app/bin/jenkinsfile-runner -w /usr/share/jenkins/ -f /workspace/Jenkinsfile
java.lang.RuntimeException: Unhandled exception
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoClassDefFoundError: hudson/model/RootAction
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructor0(Class.java:3075)
at java.lang.Class.newInstance(Class.java:412)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
... 10 more
Caused by: java.lang.ClassNotFoundException: hudson.model.RootAction
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 38 more
root@c8ef72baba98:/#
At this point I thought maybe it was a difference in Java?
Java included in jenkins:lts
root@c8ef72baba98:/# java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
java included with jenkinsfile-runner
shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile --entrypoint /bin/bash -it jenkins/jenkinsfile-runner:latest
root@4c6eeb3bb2e1:/# java -version
openjdk version "1.8.0_262"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_262-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.262-b10, mixed mode)
Seems like the java thing is probably the issue?
Looks like the unpacked Jenkins core is not present in the class path. No idea why, need to take a look
@oleg-nenashev Maybe you can direct me? Are you talking about /app
or /usr/share/jenkins
This is what I see:
root@7c9240dff52f:/usr/share/jenkins# ls -la
total 64700
drwxr-xr-x 1 root root 4096 May 28 2020 .
drwxr-xr-x 1 root root 4096 May 28 2020 ..
-rw-r--r-- 1 root root 66239216 May 28 2020 jenkins.war
drwxr-xr-x 1 jenkins root 4096 May 28 2020 ref
Note that in my Dockerfile
, I never exploded any war.
When I do explode it... I don't get very much
jar -xvf jenkins.war
created: META-INF/
inflated: META-INF/MANIFEST.MF
created: ref/
created: ref/init.groovy.d/
It seemed strange for it to be that empty, maybe I ran the jar
command with the wrong flags?
Anyways, he is how the .war
file gets put their by the upstream Dockerfile
:
# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war
# could use ADD but this one does not check Last-Modified header neither does it allow to control checksum
# see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
&& echo "${JENKINS_SHA} /usr/share/jenkins/jenkins.war" | sha256sum -c -
So it seems like it should be the correct war?
But I'm still getting the same error, even after exploding the war.
Also the contents of the /app
directory, which look correct.
root@7c9240dff52f:/usr/share/jenkins# ls /app/*
/app/bin:
jenkins-plugin-manager.jar jenkinsfile-runner jenkinsfile-runner-launcher jenkinsfile-runner.bat
/app/jenkins:
ColorFormatter.class JNLPMain.class LogFileOutputStream$1.class LogFileOutputStream$2.class LogFileOutputStream.class META-INF Main$FileAndDescription.class Main.class MainDialog$1$1.class MainDialog$1.class MainDialog.class WEB-INF bootstrap executable favicon.ico robots.txt winstone.jar
/app/lib:
payload setup
/app/repo:
com commons-collections commons-io info io org
Okay so the contents for /app/jenkins
is the exploded war from the JFR docker image.
RUN mkdir /app && unzip /jenkinsfile-runner/vanilla-package/target/war/jenkins.war -d /app/jenkins && \
rm -rf /app/jenkins/scripts /app/jenkins/jsbundles /app/jenkins/css /app/jenkins/images /app/jenkins/help /app/jenkins/WEB-INF/detached-plugins /app/jenkins/WEB-INF/jenkins-cli.jar /app/jenkins/WEB-INF/lib/jna-4.5.2.jar \
but what I can't figure out is why is the war from the jenkins:lts
image so empty?
I don't know what I did wrong... but I deleted the container and spun it back up fresh and I could properly explode /usr/share/jenkins/jenkins.war
and it had tons of files in it.
But I still get error:
root@a809acf3a116:/usr/share/jenkins# /app/bin/jenkinsfile-runner -w /usr/share/jenkins -f /workspace/Jenkinsfile
java.lang.RuntimeException: Unhandled exception
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoSuchFieldError: SYSTEM2
at io.jenkins.jenkinsfile.runner.JenkinsfileRunnerLauncher.doLaunch(JenkinsfileRunnerLauncher.java:30)
at io.jenkins.jenkinsfile.runner.JenkinsLauncher.launch(JenkinsLauncher.java:121)
at io.jenkins.jenkinsfile.runner.App.run(App.java:32)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
... 10 more
Is this because my container is using OpenJDK
and JFR is expecting AdoptOpenJDK
?
Strangely, when switching from jenkins:lts
which is jdk8
to jenkins/jenkins:latest-jdk11
, I get a different error 🤔 :
root@e239ad31accb:/# /app/bin/jenkinsfile-runner -w /usr/share/jenkins -f workspace/Jenkinsfile
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/usr/share/jenkins/WEB-INF/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
java.lang.RuntimeException: Unhandled exception
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoClassDefFoundError: com/cloudbees/plugins/credentials/CredentialsUnavailableException
at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
at java.base/java.lang.Class.getMethod0(Class.java:3293)
at java.base/java.lang.Class.getMethod(Class.java:2106)
at io.jenkins.jenkinsfile.runner.JenkinsfileRunnerLauncher.doLaunch(JenkinsfileRunnerLauncher.java:34)
at io.jenkins.jenkinsfile.runner.JenkinsLauncher.launch(JenkinsLauncher.java:121)
at io.jenkins.jenkinsfile.runner.App.run(App.java:32)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
... 10 more
Caused by: java.lang.ClassNotFoundException: com.cloudbees.plugins.credentials.CredentialsUnavailableException
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 20 more
root@e239ad31accb:/# java -version
openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment Temurin-11.0.13+8 (build 11.0.13+8)
OpenJDK 64-Bit Server VM Temurin-11.0.13+8 (build 11.0.13+8, mixed mode)
My job doesn't use credentials? If that plugin required for use by JFR?
Last update for the night. The credentials plugin is installed, I have no clue why I am getting that error 😦.
I did fix the no Jenkinsfile in current directory.
error. I tried passing a single string with a space in it, but it needed to be broken up like this:
CMD ["-f", "/workspace/Jenkinsfile"]
I can now call it from the cmd line with out having to enter the container and run the command manually.
shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile jsl_jfr
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/usr/share/jenkins/WEB-INF/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
java.lang.RuntimeException: Unhandled exception
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoClassDefFoundError: com/cloudbees/plugins/credentials/CredentialsUnavailableException
at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
at java.base/java.lang.Class.getMethod0(Class.java:3293)
at java.base/java.lang.Class.getMethod(Class.java:2106)
at io.jenkins.jenkinsfile.runner.JenkinsfileRunnerLauncher.doLaunch(JenkinsfileRunnerLauncher.java:34)
at io.jenkins.jenkinsfile.runner.JenkinsLauncher.launch(JenkinsLauncher.java:121)
at io.jenkins.jenkinsfile.runner.App.run(App.java:32)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
... 10 more
Caused by: java.lang.ClassNotFoundException: com.cloudbees.plugins.credentials.CredentialsUnavailableException
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 20 more
Still not sure why I'm getting a ton of weird errors, but I'm getting the closest I have been is with jdk11.
SERIOUSLY THE LAST UPDATE TONIGHT.
Do you think if I passed the -p
flag... then this would work correctly 👀
shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile jsl_jfr
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/usr/share/jenkins/WEB-INF/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Started
Resume disabled by user, switching to high-performance, low-durability mode.
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /tmp/jenkinsfileRunner.tmp/jfr11682560948436011876.run/workspace/job
[Pipeline] {
[Pipeline] echo
working
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
We will never speak of this again 🙄
So this is the final Dockerfile
:
ARG baseImage=jenkins/jenkins:latest-jdk11
FROM jenkins/jenkinsfile-runner:latest as jfr
FROM ${baseImage}
USER root
COPY --from=jfr /app /app
# These are optional? kinda? Not sure what the bare min plugins are for JFR
RUN jenkins-plugin-cli --plugins workflow-aggregator filesystem_scm git
RUN cd /usr/share/jenkins && jar -xvf jenkins.war
ENV JENKINS_HOME="/usr/share/jenkins/ref/"
ENV JAVA_OPTS="-Djenkins.model.Jenkins.slaveAgentPort=50000 -Djenkins.model.Jenkins.slaveAgentPortEnforce=true -Dhudson.model.LoadStatistics.clock=1000"
ENTRYPOINT ["/app/bin/jenkinsfile-runner", "-w", "/usr/share/jenkins/", "-p", "/usr/share/jenkins/ref/plugins", "-f"]
CMD ["/workspace/Jenkinsfile"]
Expect a PR from me in the next day or two.
@oleg-nenashev with the PR merged, I think this is closable.
@shadycuz maybe. In my case I was rather thinking about broader story as mentioned in the title: "Make Jenkinsfile Runner accessible for non-Java users". I believe there are many other enhancements that could be done towards this goal, e.g. #570 or #572. Probably I should create another issue for the roadmap and rename this ticket, WDYT?
I think a GitHub project around user experience would work best.
That way each story gets it's own issue and it's easy to group things and track progress.
@oleg-nenashev
Hey everyone,
I'm a big fan of the Jenkinsfile-runner, I have been for years when I accidentally stumbled upon it in early development. Unfortunately I have never been able to get it to work. I revisit often and from time to time I see updated instructions and so I try again and again but never am I successful. Well not on my own, see I have actually used the project for a year now but only because I steal working solutions, like the dockerfile from here.
I'm sure the project is easy to build and run if your very intimate with either Jenkins/Java or both. But I feel this project would be much more successful if you could make it so anyone can run the Jenkinsfile-runner. I think a good first step would be to provide instructions to build a Jenkinsfile-runner based on the Jenkins docker image. Lots of people are already using this image at 500million pulls. This would make it extremely easy to take a companies production Jenkins and then put it in a CI/CD pipeline. Also the steps to configure the
jenkins:jenkins:lts
image is already well defined. So it should make customizing a Jenkinsfile-runner even easier.I'm not 100% on the implementation details but I think you could provide two routes.
and/or
That to me would make it so much easier for me to move people to Jenkinsfile-runner or get them started 🚀
Thanks!