search

jenkinsci / publish-over-ssh-plugin

https://plugins.jenkins.io/publish-over-ssh/
113 stars 151 forks source link

Publisher file transfer over ssh hangs at 32 kilobytes #334

Closed w1552270 closed 7 months ago

w1552270 commented 7 months ago

Jenkins and plugins versions report

Jenkins: 2.414.2 OS: Linux - 3.10.0-1160.95.1.el7.x86_64 Java: 11.0.20 - Red Hat, Inc. (OpenJDK 64-Bit Server VM)

publish-over:0.22 publish-over-ssh:1.25 ssh:2.6.1 ssh-agent:346.vda_a_c4f2c8e50 ssh-credentials:308.ve4497b_ccd8f4 ssh-slaves:2.916.vd17b_43357ce4 sshd:3.312.v1c601b_c83b_0e

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins on Centos 7 scp destination on Almalinux 9.3

Reproduction steps

attempt to copy a file of size >32 kb using ssh publisher

pipeline: sshPublisher failOnError: true, publishers:[sshPublisherDesc(configName: 'server1',transfers: [sshTransfer(execTimeout: 60000, makeEmptyDirs: true,removePrefix: 'build/', sourceFiles: 'build/*.tar.gz', remoteDirectory: "${BRANCH_NAME}", usePty: true)], verbose: true)]

Expected Results

file is copied to the destination ( this used to work on older clients on centos 7)

Actual Results

pipeline hangs, file is created on destination but stops at filesize 32629.

Exception:

PM WARNING jenkins.plugins.publish_over.BPInstanceConfig perform An exception was caught when invoking perform java.net.SocketException: Connection timed out (Write failed) at java.base/java.net.SocketOutputStream.socketWrite0(Native Method) at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150) at com.jcraft.jsch.IO.put(IO.java:69) at com.jcraft.jsch.Session._write(Session.java:1619) at com.jcraft.jsch.Session.write(Session.java:1589) at com.jcraft.jsch.ChannelSftp.sendWRITE(ChannelSftp.java:2591) at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:621) Caused: 4 at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:648) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:498) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:453) at jenkins.plugins.publish_over_ssh.BapSshClient.transferFile(BapSshClient.java:200) at jenkins.plugins.publish_over_ssh.BapSshClient.transferFile(BapSshClient.java:51) at jenkins.plugins.publish_over.BPTransfer.transferFile(BPTransfer.java:171) at jenkins.plugins.publish_over.BPTransfer.transfer(BPTransfer.java:155) at jenkins.plugins.publish_over.BPTransfer.transfer(BPTransfer.java:142) at jenkins.plugins.publish_over.BapPublisher$Performer.transfer(BapPublisher.java:276) at jenkins.plugins.publish_over.BapPublisher$Performer.perform(BapPublisher.java:232) at jenkins.plugins.publish_over.BapPublisher$Performer.access$000(BapPublisher.java:205) at jenkins.plugins.publish_over.BapPublisher.perform(BapPublisher.java:158) at jenkins.plugins.publish_over.BPCallablePublisher.invoke(BPCallablePublisher.java:65) Caused: jenkins.plugins.publish_over.BapPublisherException: Exception when publishing, exception message [java.net.SocketException: Connection timed out (Write failed)] at jenkins.plugins.publish_over.BPCallablePublisher.invoke(BPCallablePublisher.java:69) at jenkins.plugins.publish_over.BPCallablePublisher.invoke(BPCallablePublisher.java:38) at hudson.FilePath.act(FilePath.java:1198) at hudson.FilePath.act(FilePath.java:1181) at jenkins.plugins.publish_over.BPInstanceConfig.perform(BPInstanceConfig.java:141) at jenkins.plugins.publish_over.BPPlugin.perform(BPPlugin.java:126) at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123) at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101) at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)

Anything else?

destination hosts are running Almalinux 9.3 with CIS server level 2 hardening profile

w1552270 commented 7 months ago

I found the culprit was one of the two firewalls (opnsense routing to another firewall to reach the destination servers). Online found indication this could be an MTU related problem, I was not able to fix it regardless of how I configured it though. Closing cause not caused by the plugin itself.