search

jenkinsci / docker-ssh-agent

Docker image for Jenkins agents connected over SSH
https://hub.docker.com/repository/docker/jenkins/ssh-agent
MIT License
183 stars 175 forks source link

SSH jenkins user does not have permissions to write remote.jar into AGENT_WORKDIR #182

Closed tpzumezawa closed 1 year ago

tpzumezawa commented 1 year ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.361.4 OS: Linux - 6.0.6-76060006-generic --- ace-editor:1.1 ant:481.v7b_09e538fcca antisamy-markup-formatter:155.v795fb_8702324 apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61 bootstrap5-api:5.2.1-3 bouncycastle-api:2.26 branch-api:2.1051.v9985666b_f6cc build-timeout:1.25 caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:1.8.0 cloudbees-folder:6.758.vfd75d09eea_a_1 command-launcher:90.v669d7ccb_7c31 commons-lang3-api:3.12.0-36.vd97de6465d5b_ commons-text-api:1.10.0-27.vb_fa_3896786a_7 credentials:1189.vf61b_a_5e2f62e credentials-binding:523.vd859a_4b_122e6 display-url-api:2.3.6 durable-task:501.ve5d4fc08b0be echarts-api:5.4.0-1 email-ext:2.92 font-awesome-api:6.2.1-1 git:4.14.0 git-client:3.13.0 github:1.36.0 github-api:1.303-400.v35c2d8258028 github-branch-source:1696.v3a_7603564d04 gradle:2.1.1 instance-identity:116.vf8f487400980 ionicons-api:31.v4757b_6987003 jackson2-api:2.13.4.20221013-295.v8e29ea_354141 jakarta-activation-api:2.0.1-2 jakarta-mail-api:2.0.1-2 javax-activation-api:1.2.0-5 javax-mail-api:1.6.2-8 jaxb:2.3.7-1 jdk-tool:63.v62d2fd4b_4793 jjwt-api:0.11.5-77.v646c772fddb_0 jquery3-api:3.6.1-2 jsch:0.1.55.61.va_e9ee26616e7 junit:1160.vf1f01a_a_ea_b_7f ldap:2.12 mailer:438.v02c7f0a_12fa_4 matrix-auth:3.1.5 matrix-project:785.v06b_7f47b_c631 mina-sshd-api-common:2.9.2-50.va_0e1f42659a_a mina-sshd-api-core:2.9.2-50.va_0e1f42659a_a momentjs:1.1.1 okhttp-api:4.9.3-108.v0feda04578cf pam-auth:1.10 pipeline-build-step:2.18 pipeline-github-lib:38.v445716ea_edda_ pipeline-graph-analysis:195.v5812d95a_a_2f9 pipeline-groovy-lib:621.vb_44ce045b_582 pipeline-input-step:456.vd8a_957db_5b_e9 pipeline-milestone-step:101.vd572fef9d926 pipeline-model-api:2.2118.v31fd5b_9944b_5 pipeline-model-definition:2.2118.v31fd5b_9944b_5 pipeline-model-extensions:2.2118.v31fd5b_9944b_5 pipeline-rest-api:2.27 pipeline-stage-step:296.v5f6908f017a_5 pipeline-stage-tags-metadata:2.2118.v31fd5b_9944b_5 pipeline-stage-view:2.27 plain-credentials:139.ved2b_9cf7587b plugin-util-api:2.18.0 popper2-api:2.11.6-2 resource-disposer:0.20 scm-api:621.vda_a_b_055e58f7 script-security:1218.v39ca_7f7ed0a_c snakeyaml-api:1.33-90.v80dcb_3814d35 ssh-agent:295.v9ca_a_1c7cc3a_a_ ssh-credentials:305.v8f4381501156 ssh-slaves:2.854.v7fd446b_337c9 sshd:3.249.v2dc2ea_416e33 structs:324.va_f5d6774f3a_d timestamper:1.21 token-macro:321.vd7cc1f2a_52c8 trilead-api:2.84.v72119de229b_7 variant:59.vf075fe829ccb workflow-aggregator:590.v6a_d052e5a_a_b_5 workflow-api:1200.v8005c684b_a_c6 workflow-basic-steps:994.vd57e3ca_46d24 workflow-cps:3536.vb_8a_6628079d5 workflow-durable-task-step:1217.v38306d8fa_b_5c workflow-job:1254.v3f64639b_11dd workflow-multibranch:716.vc692a_e52371b_ workflow-scm-step:400.v6b_89a_1317c9a_ workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:839.v35e2736cfd5c ws-cleanup:0.43 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Host is ubuntu 22.04 using docker images version: server: jenkins/jenkins:lts-jdk11 agent: jenkins/ssh-agent:jdk11

Reproduction steps

Run docker compose and setup jenkins server Add proper ssh credential to jenkins server and jenkins agent Add new ssh agent on the jenkins server Sever wont be able to copy remote.jar file because it does not have permissions for /home/jenkins/agent

Docker compose file: version: "3.9"

lts-jdk11

services: jenkins: image: jenkins/jenkins:lts-jdk11 container_name: jenkins-server privileged: true hostname: jenkinsserver user: root ports:

volumes: jenkins-data:

Expected Results

Jenkins agent should be added to the nodes on jenkins server

Actual Results

SSHLauncher{host='agent', port=22, credentialsId='jenk-slave-cred', jvmOptions='', javaPath='', prefixStartSlaveCmd='', suffixStartSlaveCmd='', launchTimeoutSeconds=60, maxNumRetries=10, retryWaitTime=15, sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.NonVerifyingKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true} [11/23/22 19:09:26] [SSH] Opening SSH connection to agent:22. [11/23/22 19:09:26] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection. [11/23/22 19:09:26] [SSH] Authentication successful. [11/23/22 19:09:26] [SSH] The remote user's environment is: AGENT_WORKDIR=/home/jenkins/agent BASH=/bin/bash BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath BASH_ALIASES=() BASH_ARGC=([0]="0") BASH_ARGV=() BASH_CMDS=() BASH_EXECUTION_STRING=set BASH_LINENO=() BASH_SOURCE=() BASH_VERSINFO=([0]="5" [1]="1" [2]="4" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu") BASH_VERSION='5.1.4(1)-release' DIRSTACK=() EUID=1000 GROUPS=() HOME=/home/jenkins HOSTNAME=08f478e63945 HOSTTYPE=x86_64 IFS=$' \t\n' JAVA_HOME=/opt/java/openjdk JENKINS_AGENT_HOME=/home/jenkins JENKINS_AGENT_SSH_PUBKEY='ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ8qhkKLoyV3rRT++FGjgRVQHuUXx/Ly1hQAfzLoT/fa umezawa@yoko' LC_ALL=C.UTF-8 LOGNAME=jenkins MACHTYPE=x86_64-pc-linux-gnu MOTD_SHOWN=pam OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PIPESTATUS=([0]="0") PPID=20 PS4='+ ' PWD=/home/jenkins SHELL=/bin/bash SHELLOPTS=braceexpand:hashall:interactive-comments SHLVL=1 SSH_CLIENT='172.18.0.3 50066 22' SSHCONNECTION='172.18.0.3 50066 172.18.0.2 22' TERM=dumb UID=1000 USER=jenkins =']' Checking Java version in the PATH openjdk version "11.0.16.1" 2022-08-12 OpenJDK Runtime Environment Temurin-11.0.16.1+1 (build 11.0.16.1+1) OpenJDK 64-Bit Server VM Temurin-11.0.16.1+1 (build 11.0.16.1+1, mixed mode) [11/23/22 19:09:26] [SSH] Checking java version of /home/jenkins/agent/jdk/bin/java Couldn't figure out the Java version of /home/jenkins/agent/jdk/bin/java bash: line 1: /home/jenkins/agent/jdk/bin/java: No such file or directory

[11/23/22 19:09:26] [SSH] Checking java version of java [11/23/22 19:09:26] [SSH] java -version returned 11.0.16.1. [11/23/22 19:09:26] [SSH] Starting sftp client. [11/23/22 19:09:26] [SSH] Copying latest remoting.jar... java.io.IOException: Could not copy remoting.jar into '/home/jenkins/agent' on agent at hudson.plugins.sshslaves.SSHLauncher.copyAgentJar(SSHLauncher.java:733) at hudson.plugins.sshslaves.SSHLauncher.lambda$launch$0(SSHLauncher.java:456) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.io.IOException: Could not copy remoting.jar to '/home/jenkins/agent/remoting.jar' on agent at hudson.plugins.sshslaves.SSHLauncher.copyAgentJar(SSHLauncher.java:725) ... 5 more Caused by: com.trilead.ssh2.SFTPException: Permission denied (SSH_FX_PERMISSION_DENIED: The user does not have sufficient permissions to perform the operation.) at com.trilead.ssh2.SFTPv3Client.openFile(SFTPv3Client.java:1201) at com.trilead.ssh2.SFTPv3Client.createFile(SFTPv3Client.java:1074) at com.trilead.ssh2.SFTPv3Client.createFile(SFTPv3Client.java:1055) at com.trilead.ssh2.jenkins.SFTPClient.writeToFile(SFTPClient.java:102) at hudson.plugins.sshslaves.SSHLauncher.copyAgentJar(SSHLauncher.java:718) ... 5 more [11/23/22 19:09:26] Launch failed - cleaning up connection [11/23/22 19:09:26] [SSH] Connection closed.

Anything else?

A work around was to modify the setup-sshd file to modify the permissions for the directory /home/jenkins/agent to allow for the "jenkins" user to write files to the agent

dduportal commented 1 year ago

I confirm I can reproduce:

docker run --pull --rm --entrypoint='' jenkins/ssh-agent:jdk11 ls -al /home/jenkins
total 32
drwxr-xr-x 1 jenkins jenkins 4096 Nov 24 18:06 .
drwxr-xr-x 1 root    root    4096 Nov 23 06:42 ..
-rw-r--r-- 1 jenkins jenkins  220 Mar 27  2022 .bash_logout
-rw-r--r-- 1 jenkins jenkins 3526 Mar 27  2022 .bashrc
drwxr-xr-x 2 root    root    4096 Nov 24 18:06 .jenkins
-rw-r--r-- 1 jenkins jenkins  807 Mar 27  2022 .profile
drwxr-xr-x 2 root    root    4096 Nov 24 18:06 agent

It sounds like a side effect of https://github.com/jenkinsci/docker-ssh-agent/pull/165 (I'm culprit).

Gotta check how to fix this (and add a test to avoid this coming back).

gounthar commented 1 year ago

Oopsie,

mea culpa, mea maxima culpa

😨

dduportal commented 1 year ago

The latest version 4.5.1 does not have the problem anymore:

$ docker pull jenkins/ssh-agent:4.5.1
4.5.1: Pulling from jenkins/ssh-agent
a8ca11554fce: Already exists 
98251271e084: Pull complete 
aa21aeddc18f: Pull complete 
46b250da2bcb: Pull complete 
4f4fb700ef54: Pull complete 
df502a0898b0: Pull complete 
996d7ee3f3dd: Pull complete 
39114b17d847: Pull complete 
Digest: sha256:e8cb848f04d7539c6493f097dee56561b3bf027b22c3ba78f57e777334dcaf89
Status: Downloaded newer image for jenkins/ssh-agent:4.5.1
docker.io/jenkins/ssh-agent:4.5.1
$ docker run --user=jenkins --rm --entrypoint='' jenkins/ssh-agent:4.5.1 ls -la /home/jenkins/agent
total 8
drwxr-xr-x 2 jenkins jenkins 4096 Nov 25 11:58 .
drwxr-xr-x 5 jenkins jenkins 4096 Nov 25 11:58 ..
$ docker-ssh-agent git:(fix/gh-182) docker run --volume=agent-dir:/home/jenkins/agent:rw --user=jenkins --rm --entrypoint='' jenkins/ssh-agent:4.5.1 ls -la /home/jenkins/agent
total 8
drwxr-xr-x 2 jenkins jenkins 4096 Nov 25 11:58 .
drwxr-xr-x 5 jenkins jenkins 4096 Nov 25 11:58 ..

Can you confirm it is now ok for you by closing the issue @tpzumezawa (or add a reproduction case)?

tpzumezawa commented 1 year ago

Sorry for the delay, yes it was fixed.