search

jenkinsci / docker-plugin

Jenkins cloud plugin that uses Docker
https://plugins.jenkins.io/docker-plugin/
MIT License
490 stars 318 forks source link

Client certificate doesn't provide to remote docker daemon #1042

Open rommanio opened 8 months ago

rommanio commented 8 months ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.426.2 OS: Linux - 5.10.0-21-amd64 Java: 17.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- ant:497.v94e7d9fffa_b_9 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 apache-httpcomponents-client-5-api:5.3-1.0 authentication-tokens:1.53.v1c90fd9191a_b_ bootstrap5-api:5.3.0-1 bouncycastle-api:2.29 branch-api:2.1122.v09cb_8ea_8a_724 build-timeout:1.31 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.0 cloud-stats:320.v96b_65297a_4b_b_ cloudbees-folder:6.848.ve3b_fd7839a_81 commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.10.0-68.v0d0b_c439292b_ credentials:1311.vcf0a_900b_37c2 credentials-binding:642.v737c34dea_6c2 display-url-api:2.3.9 docker-commons:439.va_3cb_0a_6a_fb_29 docker-java-api:3.3.4-86.v39b_a_5ede342c docker-plugin:1.5 durable-task:523.va_a_22cf15d5e0 echarts-api:5.4.0-5 email-ext:2.100 font-awesome-api:6.4.0-2 git:5.2.0 git-client:4.4.0 github:1.37.3.1 github-api:1.314-431.v78d72a_3fe4c3 github-branch-source:1732.v3f1889a_c475b_ gitlab-plugin:1.7.16 gradle:2.8.2 instance-identity:173.va_37c494ec4e5 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.15.2-350.v0c2f3f8fc595 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jersey2-api:2.40-1 jjwt-api:0.11.5-77.v646c772fddb_0 jquery3-api:3.7.0-1 junit:1217.v4297208a_a_b_ce ldap:694.vc02a_69c9787f mailer:463.vedf8358e006b_ matrix-auth:3.2 matrix-project:808.v5a_b_5f56d6966 mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_ mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_ okhttp-api:4.11.0-157.v6852a_a_fa_ec11 pam-auth:1.10 pipeline-build-step:505.v5f0844d8d126 pipeline-github-lib:42.v0739460cda_c4 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:685.v8ee9ed91d574 pipeline-input-step:477.v339683a_8d55e pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2144.v077a_d1928a_40 pipeline-model-definition:2.2144.v077a_d1928a_40 pipeline-model-extensions:2.2144.v077a_d1928a_40 pipeline-rest-api:2.33 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40 pipeline-stage-view:2.33 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.3.0 resource-disposer:0.23 scm-api:676.v886669a_199a_a_ script-security:1275.v23895f409fb_d snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4 ssh-credentials:308.ve4497b_ccd8f4 ssh-slaves:2.916.vd17b_43357ce4 structs:325.vcb_307d2a_2782 timestamper:1.26 token-macro:384.vf35b_f26814ec trilead-api:2.84.v72119de229b_7 variant:59.vf075fe829ccb workflow-aggregator:596.v8c21c963d92d workflow-api:1267.vd9b_a_ddd9eb_47 workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3774.v4a_d648d409ce workflow-durable-task-step:1289.v4d3e7b_01546b_ workflow-job:1342.v046651d5b_dfe workflow-multibranch:756.v891d88f2cd46 workflow-scm-step:415.v434365564324 workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:848.v5a_383b_d14921 ws-cleanup:0.45 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins LTS docker image, Debian 11 on the controller's host, Debian 12 on the docker daemon target.

Reproduction steps

docker-compose.yml file:

services:
  jenkins:
#    image: jenkins/jenkins:2.414.1-lts-jdk17
    image: jenkins/jenkins:2.426.2-lts-jdk17
    ports:
      - '127.0.0.1:8080:8080'
      - '50000:50000'
    volumes:
      - './jenkins_home:/var/jenkins_home'

Certificate created with algorithms: ed25519/SHA3-512

Log file:

Jan 14, 2024 7:03:08 AM FINE com.cloudbees.plugins.credentials.CredentialsNameProvider

named `<<<builder-hostname>>>-main` from com.cloudbees.plugins.credentials.common.StandardCredentials$NameProvider@5d884aaa

Jan 14, 2024 7:03:09 AM FINE com.cloudbees.plugins.credentials.CredentialsNameProvider

named `<<<builder-hostname>>>-main` from com.cloudbees.plugins.credentials.common.StandardCredentials$NameProvider@b334be

Jan 14, 2024 7:03:15 AM FINE com.cloudbees.plugins.credentials.CredentialsNameProvider

named `<<<builder-hostname>>>-main` from com.cloudbees.plugins.credentials.common.StandardCredentials$NameProvider@1f71e318

Jan 14, 2024 7:03:15 AM FINE com.github.dockerjava.core.command.AbstrDockerCmd exec

Cmd: 

Jan 14, 2024 7:03:15 AM FINEST com.github.dockerjava.core.exec.VersionCmdExec execute

GET: DefaultWebTarget{path=[/version], queryParams={}}

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.InternalHttpClient doExecute

ex-0000000025 preparing request execution

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.ProtocolExec execute

ex-0000000025 target auth state: UNCHALLENGED

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.ProtocolExec execute

ex-0000000025 proxy auth state: UNCHALLENGED

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.ConnectExec execute

ex-0000000025 acquiring connection with route {s}->https://<<<builder-hostname>>>.<<<domain.tld>>>:2376

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime acquireEndpoint

ex-0000000025 acquiring endpoint (3 MINUTES)

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager lease

ex-0000000025 endpoint lease request (3 MINUTES) [route: {s}->https://<<<builder-hostname>>>.<<<domain.tld>>>:2376][total available: 0; route allocated: 0 of 2147483647; total allocated: 0 of 2147483647]

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$3 get

ex-0000000025 endpoint leased [route: {s}->https://<<<builder-hostname>>>.<<<domain.tld>>>:2376][total available: 0; route allocated: 1 of 2147483647; total allocated: 1 of 2147483647]

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$3 get

ex-0000000025 acquired ep-0000000025

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime acquireEndpoint

ex-0000000025 acquired endpoint ep-0000000025

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.ConnectExec execute

ex-0000000025 opening connection {s}->https://<<<builder-hostname>>>.<<<domain.tld>>>:2376

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime connectEndpoint

ep-0000000025 connecting endpoint (60000000000 NANOSECONDS)

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager connect

ep-0000000025 connecting endpoint to https://<<<builder-hostname>>>.<<<domain.tld>>>:2376 (60000000000 NANOSECONDS)

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect

<<<builder-hostname>>>.<<<domain.tld>>> resolving remote address

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect

<<<builder-hostname>>>.<<<domain.tld>>> resolved to [<<<builder-hostname>>>.<<<domain.tld>>>/<<<builder-hostname-ipv4-address>>>]

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect

<<<builder-hostname>>>.<<<domain.tld>>>:2376 connecting null-><<<builder-hostname>>>.<<<domain.tld>>>/<<<builder-hostname-ipv4-address>>>:2376 (60000000000 NANOSECONDS)

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory connectSocket

Connecting socket to <<<builder-hostname>>>.<<<domain.tld>>>/<<<builder-hostname-ipv4-address>>>:2376 with timeout 60000000000 NANOSECONDS

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory executeHandshake

Enabled protocols: [TLSv1.3, TLSv1.2]

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory executeHandshake

Enabled cipher suites: [TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]

Jan 14, 2024 7:03:15 AM FINE org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory executeHandshake

Starting handshake (null)

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.ssl.TlsSessionValidator verifySession

Secure session established

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.ssl.TlsSessionValidator verifySession

 negotiated protocol: TLSv1.3

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.ssl.TlsSessionValidator verifySession

 negotiated cipher suite: TLS_AES_128_GCM_SHA256

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.ssl.TlsSessionValidator verifySession

 peer principal: CN=builder1.<<<domain.tld>>>

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.ssl.TlsSessionValidator verifySession

 peer alternative names: [<<<builder-hostname>>>.<<<domain.tld>>>, <<<internal-name>>>.<<<domain.tld>>>, <<<builder-hostname-ipv4-address>>>, 127.0.0.1, 0:0:0:0:0:0:0:1, 2a02:c207:2026:5586:0:0:0:1]

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.ssl.TlsSessionValidator verifySession

 issuer principal: EMAILADDRESS="SRE-infra+CA@<<<domain.tld>>>", CN=sec.<<<domain.tld>>>, OU=Docker, O=<<<Organization>>>, L=London, ST=London, C=GB

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection setSocketTimeout

http-outgoing-24 set socket timeout to 0 MILLISECONDS

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect

<<<builder-hostname>>>.<<<domain.tld>>>:2376 connected null-><<<builder-hostname>>>.<<<domain.tld>>>/<<<builder-hostname-ipv4-address>>>:2376 as http-outgoing-24

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager connect

ep-0000000025 connected http-outgoing-24

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime connectEndpoint

ep-0000000025 endpoint connected

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.classic.MainClientExec execute

ex-0000000025 executing GET /version HTTP/1.1

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.protocol.RequestAddCookies process

ex-0000000025 Cookie spec selected: strict

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection setSocketTimeout

http-outgoing-24 set socket timeout to 60000000000 NANOSECONDS

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime execute

ep-0000000025 start execution ex-0000000025

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint execute

ep-0000000025 executing exchange ex-0000000025 over http-outgoing-24

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection onRequestSubmitted

http-outgoing-24 >> GET /version HTTP/1.1

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection onRequestSubmitted

http-outgoing-24 >> accept: application/json

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection onRequestSubmitted

http-outgoing-24 >> Accept-Encoding: gzip, x-gzip, deflate

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection onRequestSubmitted

http-outgoing-24 >> Host: <<<builder-hostname>>>.<<<domain.tld>>>:2376

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection onRequestSubmitted

http-outgoing-24 >> Connection: keep-alive

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection onRequestSubmitted

http-outgoing-24 >> User-Agent: Apache-HttpClient/5.3 (Java/17.0.9)

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "GET /version HTTP/1.1[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "accept: application/json[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "Accept-Encoding: gzip, x-gzip, deflate[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "Host: <<<builder-hostname>>>.<<<domain.tld>>>:2376[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "Connection: keep-alive[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "User-Agent: Apache-HttpClient/5.3 (Java/17.0.9)[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 >> "[\r][\n]"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.Wire wire

http-outgoing-24 << "[read] I/O error: Received fatal alert: bad_certificate"

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection close

http-outgoing-24 Close connection

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime discardEndpoint

ep-0000000025 endpoint closed

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime discardEndpoint

ep-0000000025 discarding endpoint

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager release

ep-0000000025 releasing endpoint

Jan 14, 2024 7:03:16 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager release

ep-0000000025 connection released [route: {s}->https://<<<builder-hostname>>>.<<<domain.tld>>>:2376][total available: 0; route allocated: 0 of 2147483647; total allocated: 0 of 2147483647]

At the docker daemon target's side it is only following error:

tls: client didn't provide a certificate

Expected Results

Provide valid client certificate.

Actual Results

Certificate doesn't provided; it seems to be even not used. I tried to paste some symbols between -----BEGIN CERTIFICATE-----/-----BEGIN PRIVATE KEY----- and -----END CERTIFICATE-----/-----END PRIVATE KEY-----, tried remove any content between the same lines, results doesn't change.

Anything else?

No response

Are you interested in contributing a fix?

No response

rommanio commented 8 months ago

It seems similar to bug #825 in some aspects.