jenkinsci / docker-agent

Jenkins agent (base image) and inbound agent Docker images
https://hub.docker.com/r/jenkins/inbound-agent/
MIT License
282 stars 231 forks source link

NoClassDefFoundError: org/eclipse/jgit/internal/JGitText #756

Closed jesseadams closed 3 weeks ago

jesseadams commented 7 months ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.426.3 OS: Linux - 5.10.205-195.807.amzn2.x86_64 Java: 21.0.1 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- ace-editor:1.1 amazon-ecs:1.49 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 aws-credentials:218.v1b_e9466ec5da_ aws-java-sdk-ec2:1.12.633-430.vf9a_e567a_244f aws-java-sdk-ecs:1.12.633-430.vf9a_e567a_244f aws-java-sdk-efs:1.12.633-430.vf9a_e567a_244f aws-java-sdk-minimal:1.12.633-430.vf9a_e567a_244f bootstrap5-api:5.3.2-3 bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9 branch-api:2.1144.v1425d1c3d5a_7 build-timeout:1.32 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 cloudbees-folder:6.858.v898218f3609d command-launcher:107.v773860566e2e commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.11.0-95.v22a_d30ee5d36 configuration-as-code:1775.v810dc950b_514 credentials:1319.v7eb_51b_3a_c97b_ credentials-binding:657.v2b_19db_7d6e6d display-url-api:2.200.vb_9327d658781 durable-task:550.v0930093c4b_a_6 echarts-api:5.4.3-2 font-awesome-api:6.5.1-2 git:5.2.1 git-client:4.6.0 github:1.38.0 github-api:1.318-461.v7a_c09c9fa_d63 gson-api:2.10.1-15.v0d99f670e0a_7 instance-identity:185.v303dc7c645f9 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.16.1-373.ve709c6871598 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jdk-tool:73.vddf737284550 joda-time-api:2.12.7-29.v5a_b_e3a_82269a_ jquery3-api:3.7.1-1 json-api:20240205-27.va_007549e895c json-path-api:2.9.0-33.v2527142f2e1d junit:1259.v65ffcef24a_88 mailer:463.vedf8358e006b_ matrix-project:822.824.v14451b_c0fd42 mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_ mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_ okhttp-api:4.11.0-172.vda_da_1feeb_c6e pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-groovy-lib:704.vc58b_8890a_384 pipeline-input-step:477.v339683a_8d55e pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2175.v76a_fff0a_2618 pipeline-model-definition:2.2175.v76a_fff0a_2618 pipeline-model-extensions:2.2175.v76a_fff0a_2618 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2175.v76a_fff0a_2618 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.8.0 prism-api:1.29.0-10 resource-disposer:0.23 scm-api:683.vb_16722fb_b_80b_ script-security:1321.va_73c0795b_923 snakeyaml-api:2.2-111.vc6598e30cc65 ssh-credentials:308.ve4497b_ccd8f4 sshd:3.322.v159e91f6a_550 structs:337.v1b_04ea_4df7c8 token-macro:400.v35420b_922dcb_ trilead-api:2.133.vfb_8a_7b_9c5dd1 variant:60.v7290fc0eb_b_cd workflow-aggregator:596.v8c21c963d92d workflow-api:1291.v51fd2a_625da_7 workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3853.vb_a_490d892963 workflow-durable-task-step:1331.vc8c2fed35334 workflow-job:1385.vb_58b_86ea_fff1 workflow-multibranch:773.vc4fe1378f1d5 workflow-scm-step:415.v434365564324 workflow-step-api:657.v03b_e8115821b_ workflow-support:865.v43e78cc44e0d ws-cleanup:0.45 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Amazon Linux - 5.10.205-195.807.amzn2.x86_64

Reproduction steps

  1. Create a pipeline job that does a git clone
  2. Run the job

Expected Results

The job runs successfully or otherwise errors out with a command issue.

Actual Results

Exception in thread "Thread-5" java.lang.NoClassDefFoundError: org/eclipse/jgit/internal/JGitText

Anything else?

It works fine with a freestyle job with no usage of git.

Are you interested in contributing a fix?

No response

jesseadams commented 7 months ago

I was able to work around this issue by switching from tag latest-alpine-jdk21 to latest-jdk21 and building a custom image as follows.

FROM jenkins/inbound-agent:latest-jdk21

USER root
RUN apt-get update && apt-get install -y libjgit-java

Are the git jars not supposed to be on the agent by default?

MarkEWaite commented 7 months ago

Are the git jars not supposed to be on the agent by default?

The JGit jar files that you installed are not used by Jenkins (as far as I know). Jenkins downloads files from the controller to the agent when it needs them. A report of a missing internal JGit class likely means there is some other issue.

jesseadams commented 7 months ago

@MarkEWaite That is good to know. I'll dive through logs of the controller and see if anything pops out. If I can't find anything do you have any recommendations on how to ascertain additional details about what is going on (i.e. increased log verbosity, etc.)?

Is the same true for binary dependencies, such as the Docker Pipeline plugin needing the docker binary installed on the agent?

dduportal commented 7 months ago

@jesseadams as there seem to be nothing "obvious and easy" to catch, I suggest narrowing the scope. Usually, trying to reproduce with a docker-compose.yml (or vagrant or whatever system) setup would help in that matter so we could also see a complete setup.

I'll dive through logs of the controller and see if anything pops out. If I can't find anything do you have any recommendations on how to ascertain additional details about what is going on (i.e. increased log verbosity, etc.)?

Usually, checking the controller logs is a really good start. Filtering on the time when the error happens helps a lot. Also, sharing the full "raw" pipeline log output would help.

Is the same true for binary dependencies, such as the Docker Pipeline plugin needing the docker binary installed on the agent?

Not at all: the Docker Pipeline plugin uses a java Docker client (instead of expecting the docker binary). However you must have the Unix socket available. Please note that running Docker Jenkins agents on the same machine as the controller is not safe (as Docker gives full root access to the underlying OS unless you fine-tune it, it means agents could get access to the controller JENKINS_HOME).

jesseadams commented 7 months ago

Not at all: the Docker Pipeline plugin uses a java Docker client (instead of expecting the docker binary). However you must have the Unix socket available. Please note that running Docker Jenkins agents on the same machine as the controller is not safe (as Docker gives full root access to the underlying OS unless you fine-tune it, it means agents could get access to the controller JENKINS_HOME).

This isn't our experience. With this Jenkinsfile:

node {
    stage('Node Test') {
        docker.image('node:lts-alpine').inside {
            sh 'node -v'
        }
    }
}

I get this error:

[Pipeline] Start of Pipeline
[Pipeline] node
Running on [Jenkins](https://example.com/computer/(built-in)/) in /var/jenkins_home/workspace/jesse-docker-test
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Node Test)
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . node:lts-alpine
/var/jenkins_home/workspace/jesse-docker-test@tmp/durable-854b69b5/script.sh.copy: line 1: docker: not found
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker pull node:lts-alpine
/var/jenkins_home/workspace/jesse-docker-test@tmp/durable-c38d7534/script.sh.copy: line 1: docker: not found
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURE

Using this plugin @ 572.v950f58993843: https://plugins.jenkins.io/docker-workflow/

jesseadams commented 7 months ago

When I try to do a git clone using an ECS build agent without installing the jgit library on it then it fails.

Logs from the controller (LTS 2.426.3)

Feb 16, 2024 7:38:01 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud provision
Asked to provision 1 agent(s) for: ecs
Feb 16, 2024 7:38:01 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud provision
Will provision example-nonprod-ecs-rjwbs, for label: ecs
Feb 16, 2024 7:38:11 PM WARNING org.jenkinsci.remoting.util.AnonymousClassWarnings warn
Attempt to (de-)serialize synthetic class com.cloudbees.jenkins.plugins.amazonecs.ECSService$$Lambda/0x0000000100bcc000 in file:/var/jenkins_home/plugins/amazon-ecs/WEB-INF/lib/amazon-ecs.jar; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Feb 16, 2024 7:38:11 PM INFO hudson.slaves.NodeProvisioner update
example-nonprod-ecs-rjwbs provisioning successfully completed. We have now 2 computer(s)
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Match on tags: false
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Match on container definition: false
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Match on volumes: true
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Match on task role: true
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Match on execution role: true
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Match on network mode: true
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplate
Created Task Definition: {TaskDefinitionArn: arn:aws:ecs:us-east-2:058264214610:task-definition/example-nonprod-agent:4,ContainerDefinitions: [{Name: example-nonprod-agent,Image: jenkins/inbound-agent:latest-alpine-jdk21,Cpu: 1024,MemoryReservation: 2048,Links: [],PortMappings: [],Essential: true,EntryPoint: [],Command: [],Environment: [],EnvironmentFiles: [],MountPoints: [],VolumesFrom: [],Secrets: [],DependsOn: [],Privileged: false,DnsServers: [],DnsSearchDomains: [],ExtraHosts: [],DockerSecurityOptions: [],Ulimits: [],LogConfiguration: {LogDriver: awslogs,Options: {awslogs-group=example-jenkins, awslogs-region=us-east-2, awslogs-stream-prefix=jenkins-agent},SecretOptions: []},SystemControls: [],ResourceRequirements: [],CredentialSpecs: []}],Family: example-nonprod-agent,TaskRoleArn: arn:aws:iam::058264214610:role/ecs-agent,ExecutionRoleArn: arn:aws:iam::058264214610:role/ecs-execution,NetworkMode: awsvpc,Revision: 4,Volumes: [],Status: ACTIVE,RequiresAttributes: [{Name: com.amazonaws.ecs.capability.logging-driver.awslogs,}, {Name: ecs.capability.execution-role-awslogs,}, {Name: com.amazonaws.ecs.capability.docker-remote-api.1.19,}, {Name: com.amazonaws.ecs.capability.docker-remote-api.1.21,}, {Name: com.amazonaws.ecs.capability.task-iam-role,}, {Name: com.amazonaws.ecs.capability.docker-remote-api.1.18,}, {Name: ecs.capability.task-eni,}],PlacementConstraints: [],Compatibilities: [EC2, FARGATE],RuntimePlatform: {CpuArchitecture: X86_64,OperatingSystemFamily: LINUX},RequiresCompatibilities: [FARGATE],Cpu: 1024,Memory: 2048,InferenceAccelerators: [],RegisteredAt: Fri Feb 16 19:38:13 GMT 2024,RegisteredBy: arn:aws:sts::058264214610:assumed-role/ecs-task/889606a30fac45dea6d614b7ba243acb,}
Feb 16, 2024 7:38:13 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher runECSTask
[example-nonprod-ecs-rjwbs]: Starting agent with task definition arn:aws:ecs:us-east-2:058264214610:task-definition/example-nonprod-agent:4}
Feb 16, 2024 7:38:14 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher runECSTask
[example-nonprod-ecs-rjwbs]: Agent started with task arn : arn:aws:ecs:us-east-2:058264214610:task/example-nonprod/c6b5fc79ed504309a8e2b62e5decb867
Feb 16, 2024 7:38:14 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher launchECSTask
[example-nonprod-ecs-rjwbs]: TaskArn: arn:aws:ecs:us-east-2:058264214610:task/example-nonprod/c6b5fc79ed504309a8e2b62e5decb867
Feb 16, 2024 7:38:14 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher launchECSTask
[example-nonprod-ecs-rjwbs]: TaskDefinitionArn: arn:aws:ecs:us-east-2:058264214610:task-definition/example-nonprod-agent:4
Feb 16, 2024 7:38:14 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher launchECSTask
[example-nonprod-ecs-rjwbs]: ClusterArn: arn:aws:ecs:us-east-2:058264214610:cluster/example-nonprod
Feb 16, 2024 7:38:14 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher launchECSTask
[example-nonprod-ecs-rjwbs]: ContainerInstanceArn: null
Feb 16, 2024 7:38:31 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher launchECSTask
[example-nonprod-ecs-rjwbs]: Task started, waiting for agent to become online
Feb 16, 2024 7:38:31 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher waitForAgent
[example-nonprod-ecs-rjwbs]: Waiting for agent to connect
Feb 16, 2024 7:38:32 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher waitForAgent
[example-nonprod-ecs-rjwbs]: Waiting for agent to connect
Feb 16, 2024 7:38:33 PM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run
Accepted JNLP4-connect connection #1 from /10.0.5.180:53866
Feb 16, 2024 7:38:33 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher waitForAgent
[example-nonprod-ecs-rjwbs]: Waiting for agent to connect
Feb 16, 2024 7:38:34 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher waitForAgent
[example-nonprod-ecs-rjwbs]: Waiting for agent to connect
Feb 16, 2024 7:38:35 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher waitForAgent
[example-nonprod-ecs-rjwbs]: Waiting for agent to connect
Feb 16, 2024 7:38:36 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher waitForAgent
[example-nonprod-ecs-rjwbs]: Agent connected
Feb 16, 2024 7:38:36 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSComputer taskAccepted
[example-nonprod-ecs-rjwbs]: JobName: part of java-api-test #21
Feb 16, 2024 7:38:36 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSComputer taskAccepted
[example-nonprod-ecs-rjwbs]: JobUrl: job/java-api-test/21/
Feb 16, 2024 7:38:46 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSSlave _terminate
[example-nonprod-ecs-rjwbs]: Stopping: TaskArn arn:aws:ecs:us-east-2:058264214610:task/example-nonprod/c6b5fc79ed504309a8e2b62e5decb867, ClusterArn arn:aws:ecs:us-east-2:058264214610:cluster/example-nonprod
Feb 16, 2024 7:38:46 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService stopTask
Delete ECS agent task: arn:aws:ecs:us-east-2:058264214610:task/example-nonprod/c6b5fc79ed504309a8e2b62e5decb867
Feb 16, 2024 7:38:46 PM INFO jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
Computer.threadPoolForRemoting [#12] for example-nonprod-ecs-rjwbs terminated: java.nio.channels.ClosedChannelException
Feb 16, 2024 7:38:46 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSComputerListener onOffline
example-nonprod-ecs-rjwbs is offline. Cause: Connection was broken
Feb 16, 2024 7:38:46 PM INFO hudson.remoting.Request$2 run
Failed to send back a reply to the request RPCRequest:hudson.remoting.RemoteClassLoader$IClassLoader.fetch3[java.lang.String](2): hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@381edb88:JNLP4-connect connection from ip-10-0-5-180.us-east-2.compute.internal/10.0.5.180:53866": channel is already closed

Logs from build agent (docker image: jenkins/inbound-agent:latest-alpine-jdk21)

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   timestamp   |                                                                                                                                                    message                                                                                                                                                    |
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1708112311135 | WARNING: Providing the secret and agent name as positional arguments is deprecated; use "-secret" and "-name" instead.                                                                                                                                                                                        |
| 1708112311335 | Feb 16, 2024 7:38:31 PM hudson.remoting.Launcher createEngine                                                                                                                                                                                                                                                 |
| 1708112311335 | INFO: Setting up agent: example-nonprod-ecs-rjwbs                                                                                                                                                                                                                                                            |
| 1708112311347 | Feb 16, 2024 7:38:31 PM hudson.remoting.Engine startEngine                                                                                                                                                                                                                                                    |
| 1708112311347 | INFO: Using Remoting version: 3206.vb_15dcf73f6a_9                                                                                                                                                                                                                                                            |
| 1708112311347 | Feb 16, 2024 7:38:31 PM hudson.remoting.Engine startEngine                                                                                                                                                                                                                                                    |
| 1708112311347 | WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars                                                                                                                                                                                                         |
| 1708112311731 | Feb 16, 2024 7:38:31 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112311731 | INFO: Locating server among [https://jenkins.example.example.com/]                                                                                                                                                                                                                                     |
| 1708112312929 | Feb 16, 2024 7:38:32 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve                                                                                                                                                                                                                       |
| 1708112312929 | INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]                                                                                                                                                                                                                                  |
| 1708112312932 | Feb 16, 2024 7:38:32 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve                                                                                                                                                                                                                       |
| 1708112312932 | INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check                                                                                                                                                                                                   |
| 1708112312941 | Feb 16, 2024 7:38:32 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112312941 | INFO: Agent discovery successful                                                                                                                                                                                                                                                                              |
| 1708112312941 |   Agent address: jenkins-controller.example-jenkins                                                                                                                                                                                                                                                          |
| 1708112312941 |   Agent port:    50000                                                                                                                                                                                                                                                                                        |
| 1708112312941 |   Identity:      9a:a9:ba:f1:08:1c:86:7e:ab:6b:30:8e:64:f5:db:9d                                                                                                                                                                                                                                              |
| 1708112312943 | Feb 16, 2024 7:38:32 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112312943 | INFO: Handshaking                                                                                                                                                                                                                                                                                             |
| 1708112312954 | Feb 16, 2024 7:38:32 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112312954 | INFO: Connecting to jenkins-controller.example-jenkins:50000                                                                                                                                                                                                                                                 |
| 1708112313015 | Feb 16, 2024 7:38:33 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112313015 | INFO: Server reports protocol JNLP4-connect-proxy not supported, skipping                                                                                                                                                                                                                                     |
| 1708112313016 | Feb 16, 2024 7:38:33 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112313016 | INFO: Trying protocol: JNLP4-connect                                                                                                                                                                                                                                                                          |
| 1708112313047 | Feb 16, 2024 7:38:33 PM org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader run                                                                                                                                                                                                                       |
| 1708112313047 | INFO: Waiting for ProtocolStack to start.                                                                                                                                                                                                                                                                     |
| 1708112313473 | Feb 16, 2024 7:38:33 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112313473 | INFO: Remote identity confirmed: 9a:a9:ba:f1:08:1c:86:7e:ab:6b:30:8e:64:f5:db:9d                                                                                                                                                                                                                              |
| 1708112313525 | Feb 16, 2024 7:38:33 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112313525 | INFO: Connected                                                                                                                                                                                                                                                                                               |
| 1708112326377 | Feb 16, 2024 7:38:46 PM hudson.remoting.Launcher$CuiListener status                                                                                                                                                                                                                                           |
| 1708112326377 | INFO: Terminated                                                                                                                                                                                                                                                                                              |
| 1708112326396 | Feb 16, 2024 7:38:46 PM hudson.remoting.Request$2 run                                                                                                                                                                                                                                                         |
| 1708112326396 | INFO: Failed to send back a reply to the request UserRequest:hudson.Launcher$RemoteLauncher$KillTask@13159a2d: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@302192c0:JNLP4-connect connection to jenkins-controller.example-jenkins/10.0.3.114:50000": channel is already closed |
| 1708112327422 | Exception in thread "Thread-7" java.lang.NoClassDefFoundError: org/eclipse/jgit/internal/JGitText                                                                                                                                                                                                             |
| 1708112327422 |  at org.eclipse.jgit.internal.util.ShutdownHook.cleanup(ShutdownHook.java:85)                                                                                                                                                                                                                                 |
| 1708112327422 |  at java.base/java.lang.Thread.run(Unknown Source)                                                                                                                                                                                                                                                            |
| 1708112327422 | Caused by: java.lang.ClassNotFoundException: org.eclipse.jgit.internal.JGitText                                                                                                                                                                                                                               |
| 1708112327422 |  ... 2 more                                                                                                                                                                                                                                                                                                   |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MarkEWaite commented 7 months ago

I'm unable to duplicate the failure with my configuration. I use an Alpine Linux agent with jenkins/inbound-agent:alpine-jdk21 connected to a Jenkins 2.426.3 controller running on Java 21.0.2 with your list of plugins installed.

I created a freestyle project that clones the git client plugin repository over https. First attempt with command line git worked as expected. Second attempt with JGit worked as expected.

Is there a major difference between agents that run on ECS and agents that are started from a Linux command line on a machine that has docker installed? Do you see the same problem if you run an agent from a Linux command line?

When I run the Alpine agent container, the Eclipse JGit jar file is downloaded into the remoting Jar cache directory of that container as /home/jenkins/agent/remoting/jarCache/./A4/4945A35C0B639D37682A0FDA0CB4AB.jar. Does the remoting jar cache directory of your agent on ECS include a JGit jar file, probably with a different name? I found mine with the command: grep -l -F JGitText agent/remoting/jarCache/*/*.jar

MarkEWaite commented 7 months ago

Is the same true for binary dependencies, such as the Docker Pipeline plugin needing the docker binary installed on the agent?

Command line git is a binary dependency that needs to be installed on the agent. Likewise for git-lfs, Apache Maven, and many other tools that are used

oribe-tala commented 7 months ago

It happens to me, and looks like it was a non existing credential defined on a Jenkinsfile pipeline

MarkEWaite commented 7 months ago

It happens to me, and looks like it was a non existing credential defined on a Jenkinsfile pipeline

Can you provide more details so that others can duplicate the issue that you're seeing? When I run the following Pipeline, it fails as expected with a message that the credential cannot be found. No mention of JGitText in the message:

pipeline {
    agent any
    stages {
        stage('Invalid credential') {
            steps {
                checkout scmGit(branches: [[name: 'master']],
                                userRemoteConfigs: [[credentialsId: 'does-not-exist',
                                                     url: 'https://github.com/MarkEWaite/tasks.git']])
            }
        }
    }
}
oribe-tala commented 7 months ago

I am using jenkins 2.445 with kubernetes plugin , I am working on migrate from a old jenkins installation to this new brand jenkins, I have following list of plugins installed

antisamy-markup-formatter: 162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api: 4.5.14-208.v438351942757 authentication-tokens: 1.53.v1c90fd9191a_b_ blueocean-bitbucket-pipeline: 1.27.11 blueocean-commons: 1.27.11 blueocean-config: 1.27.11 blueocean-core-js: 1.27.11 blueocean-dashboard: 1.27.11 blueocean-display-url: 2.4.2 blueocean-events: 1.27.11 blueocean-git-pipeline: 1.27.11 blueocean-github-pipeline: 1.27.11 blueocean-i18n: 1.27.11 blueocean-jwt: 1.27.11 blueocean-personalization: 1.27.11 blueocean-pipeline-api-impl: 1.27.11 blueocean-pipeline-editor: 1.27.11 blueocean-pipeline-scm-api: 1.27.11 blueocean-rest-impl: 1.27.11 blueocean-rest: 1.27.11 blueocean-web: 1.27.11 blueocean: 1.27.11 bootstrap5-api: 5.3.2-3 bouncycastle-api: 2.30.1.77-225.v26ea_c9455fd9 branch-api: 2.1148.vce12cfcdf090 caffeine-api: 3.1.8-133.v17b_1ff2e0599 checks-api: 2.0.2 cloudbees-bitbucket-branch-source: 874.v659a_b_70f5e69 cloudbees-folder: 6.921.vfb_b_224371fb_4 commons-lang3-api: 3.13.0-62.v7d18e55f51e2 commons-text-api: 1.11.0-95.v22a_d30ee5d36 configuration-as-code: 1775.v810dc950b_514 credentials-binding: 657.v2b_19db_7d6e6d credentials: 1319.v7eb_51b_3a_c97b_ display-url-api: 2.200.vb_9327d658781 durable-task: 550.v0930093c4b_a_6 echarts-api: 5.4.3-2 favorite: 2.208.v91d65b_7792a_c font-awesome-api: 6.5.1-2 git-client: 4.6.0 git: 5.2.1 github-api: 1.318-461.v7a_c09c9fa_d63 github-branch-source: 1772.va_69eda_d018d4 github: 1.38.0 gson-api: 2.10.1-15.v0d99f670e0a_7 handy-uri-templates-2-api: 2.1.8-30.v7e777411b_148 htmlpublisher: 1.32 instance-identity: 185.v303dc7c645f9 ionicons-api: 56.v1b_1c8c49374e jackson2-api: 2.16.1-373.ve709c6871598 jakarta-activation-api: 2.0.1-3 jakarta-mail-api: 2.0.1-3 javax-activation-api: 1.2.0-6 javax-mail-api: 1.6.2-9 jaxb: 2.3.9-1 jenkins-design-language: 1.27.11 jjwt-api: 0.11.5-77.v646c772fddb_0 job-dsl: 1.87 joda-time-api: 2.12.7-29.v5a_b_e3a_82269a_ jquery3-api: 3.7.1-1 json-api: 20240205-27.va_007549e895c json-path-api: 2.9.0-33.v2527142f2e1d junit: 1259.v65ffcef24a_88 kubernetes-client-api: 6.10.0-240.v57880ce8b_0b_2 kubernetes-credentials: 0.11 kubernetes: 4186.v1d804571d5d4 mailer: 463.vedf8358e006b_ matrix-project: 822.824.v14451b_c0fd42 metrics: 4.2.21-449.v6960d7c54c69 mina-sshd-api-common: 2.12.0-90.v9f7fb_9fa_3d3b_ mina-sshd-api-core: 2.12.0-90.v9f7fb_9fa_3d3b_ okhttp-api: 4.11.0-172.vda_da_1feeb_c6e pipeline-build-step: 540.vb_e8849e1a_b_d8 pipeline-graph-analysis: 202.va_d268e64deb_3 pipeline-groovy-lib: 704.vc58b_8890a_384 pipeline-input-step: 491.vb_07d21da_1a_fb_ pipeline-milestone-step: 111.v449306f708b_7 pipeline-model-api: 2.2175.v76a_fff0a_2618 pipeline-model-definition: 2.2175.v76a_fff0a_2618 pipeline-model-extensions: 2.2175.v76a_fff0a_2618 pipeline-stage-step: 305.ve96d0205c1c6 pipeline-stage-tags-metadata: 2.2175.v76a_fff0a_2618 plain-credentials: 143.v1b_df8b_d3b_e48 plugin-util-api: 3.8.0 prism-api: 1.29.0-10 pubsub-light: 1.18 scm-api: 683.vb_16722fb_b_80b_ script-security: 1326.vdb_c154de8669 snakeyaml-api: 2.2-111.vc6598e30cc65 sse-gateway: 1.26 ssh-credentials: 308.ve4497b_ccd8f4 structs: 337.v1b_04ea_4df7c8 token-macro: 400.v35420b_922dcb_ trilead-api: 2.133.vfb_8a_7b_9c5dd1 variant: 60.v7290fc0eb_b_cd workflow-aggregator: 596.v8c21c963d92d workflow-api: 1291.v51fd2a_625da_7 workflow-basic-steps: 1042.ve7b_140c4a_e0c workflow-cps: 3867.v535458ce43fd workflow-durable-task-step: 1331.vc8c2fed35334 workflow-job: 1400.v7fd111b_ec82f workflow-multibranch: 783.va_6eb_ef636fb_d workflow-scm-step: 415.v434365564324 workflow-step-api: 657.v03b_e8115821b_ workflow-support: 865.v43e78cc44e0d

since I am working on migration of an exisitng Jenkinsfile, I notice the JNLP container on my pod was failing with mentioned error, after some research I notice some of my credentials were not configured yet, I just create missing credentials and after that the errors went away.

My env block looks like this:

environment {
      GITHUB_ACCESS_TOKEN            = credentials('XXXXXXXXXXX')
      GITHUB_COMMON_CREDS            = credentials('XXXXXXXXXXX')
    }
MarkEWaite commented 3 weeks ago

This is reported as resolved in JGit 6.10.0 and later. Refer to: