Volumes not deleted after the agent has run

bigon commented 1 month ago

Jenkins and plugins versions report

Environment

```text Jenkins: 2.462.3 OS: Linux - 6.1.0-26-cloud-amd64 Java: 17.0.12 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 apache-httpcomponents-client-5-api:5.4-118.v199115451c4d asm-api:9.7.1-97.v4cc844130d97 authentication-tokens:1.119.v50285141b_7e1 bitbucket:254.v9d8ec33a_7a_42 bootstrap5-api:5.3.3-1 bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_ branch-api:2.1178.v969d9eb_c728e build-pipeline-plugin:2.0.2 build-timeout:1.33 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.2.1 cloud-stats:336.v788e4055508b_ cloudbees-folder:6.955.v81e2a_35c08d3 command-launcher:115.vd8b_301cc15d0 commons-lang3-api:3.17.0-84.vb_b_938040b_078 commons-text-api:1.12.0-129.v99a_50df237f7 configuration-as-code:1850.va_a_8c31d3158b_ copyartifact:749.vfb_dca_a_9b_6549 credentials:1384.vf0a_2ed06f9c6 credentials-binding:681.vf91669a_32e45 dark-theme:479.v661b_1b_911c01 display-url-api:2.204.vf6fddd8a_8b_e9 docker-commons:443.v921729d5611d docker-java-api:3.4.0-94.v65ced49b_a_7d5 docker-plugin:1.7.0 durable-task:577.v2a_8a_4b_7c0247 echarts-api:5.5.1-3 eddsa-api:0.3.0-4.v84c6f0f4969e email-ext:1844.v3ea_a_b_842374a_ font-awesome-api:6.6.0-2 git:5.5.2 git-client:5.0.0 gson-api:2.11.0-85.v1f4e87273c33 hashicorp-vault-plugin:371.v884a_4dd60fb_6 instance-identity:201.vd2a_b_5a_468a_a_6 ionicons-api:74.v93d5eb_813d5f jackson2-api:2.17.0-379.v02de8ec9f64c jakarta-activation-api:2.1.3-1 jakarta-mail-api:2.1.3-1 javax-activation-api:1.2.0-7 javax-mail-api:1.6.2-10 jaxb:2.3.9-1 jdk-tool:80.v8a_dee33ed6f0 job-dsl:1.89 joda-time-api:2.13.0-93.v9934da_29b_a_e9 jquery:1.12.4-1 jquery3-api:3.7.1-2 jsch:0.2.16-86.v42e010d9484b_ json-api:20240303-101.v7a_8666713110 json-path-api:2.9.0-118.v7f23ed82a_8b_8 junit:1307.vdd5b_2646279e mailer:488.v0c9639c1a_eb_3 matrix-auth:3.2.2 matrix-project:839.vff91cd7e3a_b_2 mercurial:1260.vdfb_723cdcc81 metrics:4.2.21-451.vd51df8df52ec mina-sshd-api-common:2.14.0-133.vcc091215a_358 mina-sshd-api-core:2.14.0-133.vcc091215a_358 parameterized-trigger:806.vf6fff3e28c3e pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-graph-analysis:216.vfd8b_ece330ca_ pipeline-graph-view:340.v28cecee8b_25f pipeline-groovy-lib:740.va_2701257fe8d pipeline-input-step:495.ve9c153f6067b_ pipeline-milestone-step:119.vdfdc43fc3b_9a_ pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83 pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83 pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83 pipeline-stage-step:312.v8cd10304c27a_ pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83 plain-credentials:183.va_de8f1dd5a_2b_ plugin-util-api:5.1.0 prism-api:1.29.0-17 resource-disposer:0.24 scm-api:696.v778d637b_a_762 script-security:1365.v4778ca_84b_de5 snakeyaml-api:2.3-123.v13484c65210a_ ssh-credentials:343.v884f71d78167 ssh-slaves:2.973.v0fa_8c0dea_f9f sshd:3.330.vc866a_8389b_58 structs:338.v848422169819 theme-manager:262.vc57ee4a_eda_5d timestamper:1.27 token-macro:400.v35420b_922dcb_ trilead-api:2.147.vb_73cc728a_32e variant:60.v7290fc0eb_b_cd workflow-aggregator:600.vb_57cdd26fdd7 workflow-api:1336.vee415d95c521 workflow-basic-steps:1058.vcb_fc1e3a_21a_9 workflow-cps:3975.v567e2a_1ffa_22 workflow-durable-task-step:1371.vb_7cec8f3b_95e workflow-job:1436.vfa_244484591f workflow-multibranch:795.ve0cb_1f45ca_9a_ workflow-scm-step:427.v4ca_6512e7df1 workflow-step-api:678.v3ee58b_469476 workflow-support:930.vf51d22b_ce488 ws-cleanup:0.47 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Docker running on the host:

docker version
Client:
 Version:           20.10.24+dfsg1
 API version:       1.41
 Go version:        go1.19.8
 Git commit:        297e128
 Built:             Thu May 18 08:38:34 2023
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.24+dfsg1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       5d6db84
  Built:            Thu May 18 08:38:34 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20~ds1
  GitCommit:        1.6.20~ds1-1+b1
 runc:
  Version:          1.1.5+ds1
  GitCommit:        1.1.5+ds1-1+deb12u1
 docker-init:
  Version:          0.19.0
  GitCommit:

jenkins/jenkins 2.462.3-lts-jdk17 bac101b69b63 2 weeks ago 470MB jenkins/jnlp-agent-terraform latest a0d8776cf8d0 3 hours ago 265MB

Reproduction steps

Setup the plugin with the follwing jcasc config:

clouds:
- docker:
  dockerApi:
    connectTimeout: 60
    dockerHost:
      uri: "tcp://host.docker.internal:2375"
    readTimeout: 60
  name: "Docker"
  templates:
  - connector:
      jnlp:
        jenkinsUrl: "http://host.docker.internal:8080"
    dockerTemplateBase:
      cpuPeriod: 0
      cpuQuota: 0
      extraHosts:
      - "host.docker.internal:host-gateway"
      extraHostsString: "host.docker.internal:host-gateway"
      image: "jenkins/inbound-agent:latest"
    name: "Jenkins Agent"
    pullStrategy: PULL_ALWAYS
    pullTimeout: 300
  - connector:
      jnlp:
        jenkinsUrl: "http://host.docker.internal:8080"
    dockerTemplateBase:
      cpuPeriod: 0
      cpuQuota: 0
      extraHosts:
      - "host.docker.internal:host-gateway"
      extraHostsString: "host.docker.internal:host-gateway"
      image: "jenkins/jnlp-agent-terraform:latest"
    labelString: "terraform-container"
    mode: EXCLUSIVE
    pullStrategy: PULL_ALWAYS
    pullTimeout: 300

Run a job that use one of these images
Run docker volume ls

Expected Results

There is no left over volumes

Actual Results

Left over volumes are stacking up

Anything else?

No response

Are you interested in contributing a fix?

No response

bigon commented 1 month ago

ATM the docker plugin is using /home/jenkins/workspace as the default home directory, but the new inbound images are using /home/jenkins/agent. That means that the result of the build is not in in the volume and is deleted.

But, if the home is properly set (remoteFs: "/home/jenkins/agent") all the files are being stored and the filesystem grows unbounded.

Adding an option to delete all the volumes of the container is a must

bigon commented 1 month ago

There is a removeVolumes option 🤦

krisstern commented 1 month ago

No worries @bigon!

jenkinsci / docker-plugin