Can't make plugin to work over unix socket

rkosegi commented 8 months ago

Jenkins and plugins versions report

Environment

```text Jenkins: 2.433 OS: Linux - 4.15.0-135-generic Java: 11.0.21 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- analysis-model-api:11.13.0 ansicolor:1.0.4 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 apache-httpcomponents-client-5-api:5.2.2-1.0 authentication-tokens:1.53.v1c90fd9191a_b_ badge:1.9.1 blueocean:1.27.9 blueocean-autofavorite:1.2.5 blueocean-bitbucket-pipeline:1.27.9 blueocean-commons:1.27.9 blueocean-config:1.27.9 blueocean-core-js:1.27.9 blueocean-dashboard:1.27.9 blueocean-display-url:2.4.2 blueocean-events:1.27.9 blueocean-git-pipeline:1.27.9 blueocean-github-pipeline:1.27.9 blueocean-i18n:1.27.9 blueocean-jwt:1.27.9 blueocean-personalization:1.27.9 blueocean-pipeline-api-impl:1.27.9 blueocean-pipeline-editor:1.27.9 blueocean-pipeline-scm-api:1.27.9 blueocean-rest:1.27.9 blueocean-rest-impl:1.27.9 blueocean-web:1.27.9 bootstrap5-api:5.3.2-2 bouncycastle-api:2.29 branch-api:2.1135.v8de8e7899051 build-name-setter:2.4.0 build-timeout:1.31 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 cloud-stats:320.v96b_65297a_4b_b_ cloudbees-bitbucket-branch-source:856.v04c46c86f911 cloudbees-disk-usage-simple:187.v6378d330d1d4 cloudbees-folder:6.858.v898218f3609d command-launcher:107.v773860566e2e commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.11.0-94.v3e1f4a_926e49 conditional-buildstep:1.4.3 config-file-provider:959.vcff671a_4518b_ configuration-as-code:1746.vf1673cfe690a convert-to-pipeline:1.0 copyartifact:722.v0662a_9b_e22a_c credentials:1309.v8835d63eb_d8a_ credentials-binding:642.v737c34dea_6c2 dashboard-view:2.495.v07e81500c3f2 data-tables-api:1.13.8-1 delivery-pipeline-plugin:1.4.2 description-setter:1.10 disk-usage:1.2 display-url-api:2.200.vb_9327d658781 docker-build-step:2.10 docker-commons:439.va_3cb_0a_6a_fb_29 docker-java-api:3.3.1-79.v20b_53427e041 docker-plugin:1.3.0 docker-slaves:1.0.7 docker-workflow:572.v950f58993843 downstream-build-cache:1.7 durable-task:523.va_a_22cf15d5e0 echarts-api:5.4.3-1 extended-choice-parameter:376.v2e02857547b_a_ external-monitor-job:215.v2e88e894db_f8 extra-tool-installers:113.v5d95d475e985 favorite:2.4.3 font-awesome-api:6.4.2-1 forensics-api:2.3.0 git:5.2.1 git-client:4.5.0 git-server:99.va_0826a_b_cdfa_d github:1.37.3.1 github-api:1.318-461.v7a_c09c9fa_d63 github-branch-source:1751.v90e17c48a_6a_c gitlab-plugin:1.7.16 global-variable-string-parameter:1.2 google-login:1.8 google-metadata-plugin:0.5 google-oauth-plugin:1.318.vb_39c5db_e3041 google-play-android-publisher:4.2 google-source-plugin:0.4 gradle:2.9 h2-api:11.1.4.199-12.v9f4244395f7a_ handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 hetzner-cloud:84.v8acf5510fd35 htmlpublisher:1.32 instance-identity:185.v303dc7c645f9 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.15.3-372.v309620682326 jacoco:3.3.5 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javadoc:243.vb_b_503b_b_45537 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jdk-tool:73.vddf737284550 jenkins-design-language:1.27.9 jersey2-api:2.41-133.va_03323b_a_1396 jjwt-api:0.11.5-77.v646c772fddb_0 job-dsl:1.87 jobConfigHistory:1229.v3039470161a_d join:1.21 jquery:1.12.4-1 jquery3-api:3.7.1-1 jsch:0.2.8-65.v052c39de79b_2 junit:1240.vf9529b_881428 kubernetes-cli:1.12.1 kubernetes-client-api:6.8.1-224.vd388fca_4db_3b_ kubernetes-credentials:0.11 mailer:463.vedf8358e006b_ mapdb-api:1.0.9-28.vf251ce40855d matrix-auth:3.2.1 matrix-project:818.v7eb_e657db_924 maven-plugin:3.23 metrics:4.2.18-442.v02e107157925 mina-sshd-api-common:2.11.0-86.v836f585d47fa_ mina-sshd-api-core:2.11.0-86.v836f585d47fa_ naginator:1.421.v85c11e2db_507 next-executions:239.v27031b_e26418 oauth-credentials:0.646.v02b_66dc03d2e okhttp-api:4.11.0-157.v6852a_a_fa_ec11 parameterized-trigger:787.v665fcf2a_830b_ performance:951.v5600a_c6422ed perfpublisher:8.09 pipeline-build-step:516.v8ee60a_81c5b_9 pipeline-config-history:1.6 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:689.veec561a_dee13 pipeline-input-step:477.v339683a_8d55e pipeline-maven:1362.vee39a_d4b_02b_1 pipeline-maven-api:1362.vee39a_d4b_02b_1 pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2151.ve32c9d209a_3f pipeline-model-definition:2.2151.ve32c9d209a_3f pipeline-model-extensions:2.2151.ve32c9d209a_3f pipeline-npm:155.ve2e393062d86 pipeline-rest-api:2.34 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2151.ve32c9d209a_3f pipeline-stage-view:2.34 pipeline-timeline:1.0.3 pipeline-utility-steps:2.16.0 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.6.0 prism-api:1.29.0-9 pubsub-light:1.18 rebuild:330.v645b_7df10e2a_ resource-disposer:0.23 rich-text-publisher-plugin:1.5 robot:3.4.0 role-strategy:689.v731678c3e0eb_ run-condition:1.7 scm-api:683.vb_16722fb_b_80b_ script-security:1281.v22fb_899df1a_e snakeyaml-api:2.2-111.vc6598e30cc65 sonar:2.16.1 sse-gateway:1.26 ssh:2.6.1 ssh-agent:346.vda_a_c4f2c8e50 ssh-credentials:308.ve4497b_ccd8f4 ssh-slaves:2.916.vd17b_43357ce4 sshd:3.312.v1c601b_c83b_0e structs:325.vcb_307d2a_2782 timestamper:1.26 token-macro:384.vf35b_f26814ec toolenv:1.2 trilead-api:2.84.v72119de229b_7 variant:60.v7290fc0eb_b_cd view-job-filters:369.ve0513a_a_f5524 warnings-ng:10.5.1 workflow-aggregator:596.v8c21c963d92d workflow-api:1283.v99c10937efcb_ workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3806.va_3a_6988277b_2 workflow-durable-task-step:1289.v4d3e7b_01546b_ workflow-job:1360.vc6700e3136f5 workflow-multibranch:756.v891d88f2cd46 workflow-scm-step:415.v434365564324 workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:865.v43e78cc44e0d ws-cleanup:0.45 yet-another-build-visualizer:1.16 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

OS: Ubuntu LTS Docker: 20.10.7

Reproduction steps

run jenkins controller using docker-compose:

1)

---
version: '3.2'
services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins:lts
    restart: unless-stopped
    logging:
      driver: json-file
      options:
        tag: "{{.Name}}"
    ports:
    - 8000:8000
    - 8084:8080
    - 50000:50000
    volumes:
    - /usr/bin/docker:/usr/bin/docker:ro
    - /var/run/docker.sock:/var/run/docker.sock:rw
    - /opt/jenkins/data:/var/jenkins_home

2) Configure docker-local cloud using script from this repo

Expected Results

Docker plugin to connect to docker daemon

Actual Results

Error message that plugin can't connect: Connect to unix://localhost:2375 [localhost/127.0.0.1] failed: Connection refused (Connection refused)

Anything else?

Here are detailed logs

Nov 27, 2023 10:10:26 AM FINE com.nirima.jenkins.plugins.docker.DockerContainerWatchdog execute
Checking Docker Cloud docker-local at unix:///var/run/docker.sock
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.InternalHttpClient doExecute
ex-0000000001 preparing request execution
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.ProtocolExec execute
ex-0000000001 target auth state: UNCHALLENGED
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.ProtocolExec execute
ex-0000000001 proxy auth state: UNCHALLENGED
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.ConnectExec execute
ex-0000000001 acquiring connection with route {}->unix://localhost:2375
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime acquireEndpoint
ex-0000000001 acquiring endpoint (3 MINUTES)
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager lease
ex-0000000001 endpoint lease request (3 MINUTES) [route: {}->unix://localhost:2375][total available: 0; route allocated: 0 of 2147483647; total allocated: 0 of 2147483647]
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$3 get
ex-0000000001 endpoint leased [route: {}->unix://localhost:2375][total available: 0; route allocated: 1 of 2147483647; total allocated: 1 of 2147483647]
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$3 get
ex-0000000001 acquired ep-0000000001
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime acquireEndpoint
ex-0000000001 acquired endpoint ep-0000000001
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.ConnectExec execute
ex-0000000001 opening connection {}->unix://localhost:2375
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime connectEndpoint
ep-0000000001 connecting endpoint (3000000000 NANOSECONDS)
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager connect
ep-0000000001 connecting endpoint to unix://localhost:2375 (3000000000 NANOSECONDS)
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect
localhost resolving remote address
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect
localhost resolved to [localhost/127.0.0.1]
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect
localhost:2375 connecting null->localhost/127.0.0.1:2375 (3000000000 NANOSECONDS)
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator connect
localhost:2375 connection to localhost/127.0.0.1:2375 failed (class java.net.ConnectException); terminating operation
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection close
http-outgoing-0 close connection IMMEDIATE
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime discardEndpoint
ep-0000000001 endpoint closed
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.classic.InternalExecRuntime discardEndpoint
ep-0000000001 discarding endpoint
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager release
ep-0000000001 releasing endpoint
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager release
ep-0000000001 connection is not kept alive
Nov 27, 2023 10:10:26 AM FINE org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager release
ep-0000000001 connection released [route: {}->unix://localhost:2375][total available: 0; route allocated: 0 of 2147483647; total allocated: 0 of 2147483647]
Nov 27, 2023 10:10:26 AM WARNING com.nirima.jenkins.plugins.docker.DockerContainerWatchdog retrieveContainers
Unable to retrieve list of containers available on DockerCloud [name=docker-local, dockerURI=unix:///var/run/docker.sock] while reading list of containers (showAll=true, labelFilters={com.nirima.jenkins.plugins.docker.JenkinsId=8de6877e67724162b1587711f7c0950b})
org.apache.hc.client5.http.HttpHostConnectException: Connect to unix://localhost:2375 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
    at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
    at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
    at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.base/java.net.Socket.connect(Socket.java:609)
    at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.lambda$connectSocket$0(PlainConnectionSocketFactory.java:91)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:90)
    at org.apache.hc.client5.http.socket.ConnectionSocketFactory.connectSocket(ConnectionSocketFactory.java:123)
    at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:184)
    at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
    at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
    at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
    at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
    at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
    at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:191)
Caused: java.lang.RuntimeException
    at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:195)
    at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9)
    at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228)
    at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:202)
    at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:74)
    at com.github.dockerjava.core.exec.ListContainersCmdExec.execute(ListContainersCmdExec.java:44)
    at com.github.dockerjava.core.exec.ListContainersCmdExec.execute(ListContainersCmdExec.java:15)
    at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
    at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:33)
    at com.nirima.jenkins.plugins.docker.DockerContainerWatchdog.retrieveContainers(DockerContainerWatchdog.java:263)
    at com.nirima.jenkins.plugins.docker.DockerContainerWatchdog.processCloud(DockerContainerWatchdog.java:208)
    at com.nirima.jenkins.plugins.docker.DockerContainerWatchdog.execute(DockerContainerWatchdog.java:148)
    at hudson.model.AsyncPeriodicWork.lambda$doRun$0(AsyncPeriodicWork.java:102)
    at java.base/java.lang.Thread.run(Thread.java:829)

Nov 27, 2023 10:10:26 AM INFO com.nirima.jenkins.plugins.docker.DockerContainerWatchdog execute
Not checking the list of nodes, as list of containers is known to be incomplete

I tried to connection from jenkins master to socket using curl, no issue:

curl  -s --unix-socket /var/run/docker.sock http://v1/containers/json

[{"Id":"a53f9b6632cfc9d3132af8beb6adf1b8477609970411884fce4327b31a254a6e","Names":["/jenkins"],"Image":"docker.ifne.eu/ci-master","ImageID":"sha256:477287f0c313d59508d47028a3b7c5c078cc0013bb1ddd72fbfaae1a0b781288","Command":"/usr/bin/tini -- /usr/local/bin/jenkins.sh","Created":1701068662,"Ports":[{"IP":"0.0.0.0","PrivatePort":50000,"PublicPort":50000,"Type":"tcp"},...

I was also able to attach strace and trace down actual attempt for connection:

[pid 26737] socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 801
[pid 26737] setsockopt(801, SOL_SOCKET, SO_REUSEADDR, [0], 4) = 0
[pid 26737] setsockopt(801, SOL_TCP, TCP_NODELAY, [1], 4) = 0
[pid 26737] setsockopt(801, SOL_SOCKET, SO_KEEPALIVE, [0], 4) = 0
[pid 26737] fcntl(801, F_GETFL <unfinished ...>
[pid 26737] connect(801, {sa_family=AF_INET, sin_port=htons(2375), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)

Clearly, there should be AF_UNIX, but there is AF_INET.

Are you interested in contributing a fix?

Sure, happy to help if you give me pointers.

a-st commented 8 months ago

Today we started experiencing the same issue running Jenkins 2.426.1

PaulFridrick commented 8 months ago

Same here, I had to switch to a tcp connection on 127.0.0.1:2375 to work around the issue.

I also have another connection problem with a remote tcp docker socket secured via SSL (tcp://:2376) :

Exception while provisioning for label: 'xxxx', cloud='yyyy'
java.lang.IllegalArgumentException: Invalid Proxy
    at java.base/java.net.Socket.<init>(Socket.java:122)
    at org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.createSocket(SSLConnectionSocketFactory.java:208)
    at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:158)
    at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
    at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
    at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
    at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
    at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
    at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:191)
    at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9)
    at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228)
    at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:202)
    at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:74)
    at com.github.dockerjava.core.exec.ListContainersCmdExec.execute(ListContainersCmdExec.java:44)
    at com.github.dockerjava.core.exec.ListContainersCmdExec.execute(ListContainersCmdExec.java:15)
    at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
    at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:33)
    at com.nirima.jenkins.plugins.docker.DockerCloud.countContainersInDocker(DockerCloud.java:638)
    at com.nirima.jenkins.plugins.docker.DockerCloud.canAddProvisionedAgent(DockerCloud.java:656)
    at com.nirima.jenkins.plugins.docker.DockerCloud.provision(DockerCloud.java:394)
    at io.jenkins.docker.FastNodeProvisionerStrategy.applyToCloud(FastNodeProvisionerStrategy.java:71)
    at io.jenkins.docker.FastNodeProvisionerStrategy.apply(FastNodeProvisionerStrategy.java:41)
    at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:325)
    at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:823)
    at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:92)
    at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

I don't know if this is the same bug, but the 2 issues appeared simultaneously for me. I can create another issue if necessary.

a-st commented 8 months ago

Maybe this is related to Apache HttpComponents Client 5.x API plugin? We rolled back to version 5.2.1-1.1 and the docker plugin runs as expected.

PaulFridrick commented 8 months ago

For the second issue, a workaround is to disable SSL on the remote Docker TCP socket and disable server credentials in Jenkins configuration. But this is insecure...

@a-st I will try to rollback as it seems safer, thanks !

krisstern commented 8 months ago

We will need to look into this https://plugins.jenkins.io/apache-httpcomponents-client-5-api compatibility issue to see if we could come up with a more sustainable fix, possibly over the winter holidays.

rkosegi commented 8 months ago

Maybe this is related to Apache HttpComponents Client 5.x API plugin? We rolled back to version 5.2.1-1.1 and the docker plugin runs as expected.

Confirmed this works as workaround, thanks for sharing it

rkosegi commented 8 months ago

Same here, I had to switch to a tcp connection on 127.0.0.1:2375 to work around the issue.

I also have another connection problem with a remote tcp docker socket secured via SSL (tcp://:2376) :

Exception while provisioning for label: 'xxxx', cloud='yyyy'
java.lang.IllegalArgumentException: Invalid Proxy
  at java.base/java.net.Socket.<init>(Socket.java:122)
  at org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.createSocket(SSLConnectionSocketFactory.java:208)
  at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:158)
  at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
  at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
  at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
  at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142)
  at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
  at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
  at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
  at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
  at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
  at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
  at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
  at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
  at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
  at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
  at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
  at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:191)
  at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9)
  at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228)
  at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:202)
  at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:74)
  at com.github.dockerjava.core.exec.ListContainersCmdExec.execute(ListContainersCmdExec.java:44)
  at com.github.dockerjava.core.exec.ListContainersCmdExec.execute(ListContainersCmdExec.java:15)
  at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
  at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:33)
  at com.nirima.jenkins.plugins.docker.DockerCloud.countContainersInDocker(DockerCloud.java:638)
  at com.nirima.jenkins.plugins.docker.DockerCloud.canAddProvisionedAgent(DockerCloud.java:656)
  at com.nirima.jenkins.plugins.docker.DockerCloud.provision(DockerCloud.java:394)
  at io.jenkins.docker.FastNodeProvisionerStrategy.applyToCloud(FastNodeProvisionerStrategy.java:71)
  at io.jenkins.docker.FastNodeProvisionerStrategy.apply(FastNodeProvisionerStrategy.java:41)
  at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:325)
  at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:823)
  at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:92)
  at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
  at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
  at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)

I don't know if this is the same bug, but the 2 issues appeared simultaneously for me. I can create another issue if necessary.

This has probably different root cause, seems to be related to this

strangelookingnerd commented 8 months ago

This has probably different root cause, seems to be related to this

Maintainer of jenkinsci/apache-httpcomponents-client-5-api-plugin here. I can second @rkosegi supposition. Sorry for any inconvenience caused, but this one needs to be fixed in the httpcomponents library. I'll follow-up with a release of the plugin once there are updates available.

krisstern commented 8 months ago

Thanks @strangelookingnerd for the follow-up!

itfsw commented 7 months ago

same too

strangelookingnerd commented 7 months ago

I just released 5.2.3-1.0 of the plugin which includes the latest http client dependency version with the fix.

schreeedi commented 7 months ago

Just updated to apache-httpcomponents-client-5-api:5.2.3-1.0 but it didn't fix it for me, reverted to 5.2.1-1.1 and it works

strangelookingnerd commented 7 months ago

Digging a little deeper into the issue it seems like docker-java (which is used by this plugin) is not compatible with the latest httpclient5 versions - see https://github.com/docker-java/docker-java/pull/2262

lifeofguenter commented 7 months ago

I had a slightly different error (https://issues.jenkins.io/browse/JENKINS-72415):

java.lang.IllegalArgumentException: Invalid Proxy at java.base/java.net.Socket.(Unknown Source) at org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.createSocket(SSLConnectionSocketFactory.java:208) at

but fortunately latest version (apache-httpcomponents-client-5-api:5.2.3-1.0) resolved the issue

roman-vynar commented 7 months ago

Same here, Docker socket works only with apache-httpcomponents-client-5-api 5.2.1-1.1 and broken with 5.2.3-1.0. Actually, that's a case for a month+ already... :(

Mai-Lapyst commented 6 months ago

Still dosnt works even on newest version of 5.3-1.0 of apache-httpcomponents-client-5-api, but now with a slightly different error for me when using unix:///var/run/docker.sock as the docker host uri. Downgrading to 5.2.1-1.1 seems still be the only option for now.

Log output

``` org.apache.hc.client5.http.HttpHostConnectException: Connect to unix://localhost:2375 [localhost/127.0.0.1] failed: Connection refused at java.base/sun.nio.ch.Net.pollConnect(Native Method) at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:547) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:602) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.lambda$connectSocket$0(PlainConnectionSocketFactory.java:91) at java.base/java.security.AccessController.doPrivileged(AccessController.java:569) at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:90) at org.apache.hc.client5.http.socket.ConnectionSocketFactory.connectSocket(ConnectionSocketFactory.java:123) at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:184) at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447) at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162) at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172) at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170) at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87) at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:191) Caused: java.lang.RuntimeException at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:195) at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9) at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228) at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:202) at com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:74) at com.github.dockerjava.core.exec.VersionCmdExec.execute(VersionCmdExec.java:26) at com.github.dockerjava.core.exec.VersionCmdExec.execute(VersionCmdExec.java:13) at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21) at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:33) at io.jenkins.docker.client.DockerAPI$DescriptorImpl.doTestConnection(DockerAPI.java:428) at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:397) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:409) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:78) at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:207) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:140) at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:558) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:836) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900) at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:475) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:698) at org.kohsuke.stapler.Stapler.service(Stapler.java:248) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:163) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:248) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160) at jenkins.util.HttpServletFilter$1.doFilter(HttpServletFilter.java:76) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:166) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at jenkins.ErrorAttributeFilter.doFilter(ErrorAttributeFilter.java:29) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:154) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:94) at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:54) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:110) at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:101) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:227) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:221) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:117) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:63) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:111) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:172) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:53) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:86) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:38) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:569) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.Server.handle(Server.java:563) at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) at java.base/java.lang.Thread.run(Thread.java:840) ```

ethann commented 6 months ago

Finally fixed in just released version 5.3.1-1.0 of apache-httpcomponents-client-5-api. 🎉

rkosegi commented 5 months ago

Tested with 5.3.1-1.0 with same, successful results. Thanks to everybody who participated in debugging and resolving the issue. Closing ticket now.

jenkinsci / docker-plugin