search

rundeck-plugins / rundeck-winrm-plugin

Rundeck WinRM Node Executor plugin
Other
56 stars 28 forks source link

BufferException: Underflow / trying to reach share \\[servername]\t$mp #76

Open pts-aut opened 2 years ago

pts-aut commented 2 years ago

Environment: In our Evniroment we've implemented rundeck docker container with overthere-winrm and a separate windows server 2019 machine with WinRM components configured. Until now we're using PyWinrm to dispatch the scripts to the windows node - but the file copy is much to slow, so we wanted to switch to overthere-winrm.

The issue: When overthere-winrm is trying to copy the files in the "copy file" step of the job image it fails with: Failed: PluginFailed: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow

Complete error:

[workflow] beginExecuteNodeStep(tst-node): NodeDispatch: StepExecutionItem{type='NodeDispatch', keepgoingOnSuccess=false, hasFailureHandler=false}
[overthere-winrm:WINSRV] 
[overthere-winrm:WINSRV] ConnectionOptions[
    connectionTimeoutMillis --> 15000
    winrmEnableHttps --> false
    password --> ********
    address --> WINSRV
    os --> WINDOWS
    winrmKerberosUseHttpSpn --> false
    port --> 5985
    winrmKerberosAddPortToSpn --> false
    file-copy-destination-dir --> null
    connectionType --> WINRM_INTERNAL
    username --> [username]
]
[overthere-winrm:WINSRV] Destination file acquired temp\base_functions.ps1
[overthere-winrm:WINSRV] Copying file to temp\base_functions.ps1
[overthere-winrm:WINSRV] Closing connection
Failed executing node plugin [copyfile] on node sap-tst: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
    at com.hierynomus.smbj.session.Session.connectTree(Session.java:139)
    at com.hierynomus.smbj.session.Session.connectShare(Session.java:98)
    at com.xebialabs.overthere.smb.SmbConnection.getShare(SmbConnection.java:169)
    at com.xebialabs.overthere.smb.SmbFile.getShare(SmbFile.java:345)
    at com.xebialabs.overthere.smb.SmbFile.getOutputStream(SmbFile.java:203)
    at com.dtolabs.rundeck.plugin.overthere.OTWinRMFileCopier.copyFileStream(OTWinRMFileCopier.java:109)
    at com.dtolabs.rundeck.plugin.overthere.OTWinRMFileCopier.copyFile(OTWinRMFileCopier.java:139)
    at com.dtolabs.rundeck.core.execution.service.MultiFileCopierUtil.copyMultipleFiles(MultiFileCopierUtil.java:62)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.fileCopyFiles(ExecutionServiceImpl.java:352)
    at com.dtolabs.rundeck.plugin.copyfile.CopyFileNodeStepPlugin.copyFileList(CopyFileNodeStepPlugin.java:207)
    at com.dtolabs.rundeck.plugin.copyfile.CopyFileNodeStepPlugin.executeNodeStep(CopyFileNodeStepPlugin.java:113)
    at com.dtolabs.rundeck.core.execution.workflow.steps.node.NodeStepPluginAdapter.executeNodeStep(NodeStepPluginAdapter.java:167)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeNodeStep(ExecutionServiceImpl.java:206)
    at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:130)
    at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:61)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.dispatchToNodesWith(ExecutionServiceImpl.java:262)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.dispatchToNodes(ExecutionServiceImpl.java:233)
    at com.dtolabs.rundeck.core.execution.workflow.steps.NodeDispatchStepExecutor.executeWorkflowStep(NodeDispatchStepExecutor.java:66)
    at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeStep(ExecutionServiceImpl.java:111)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowExecutor.executeWFItem(BaseWorkflowExecutor.java:285)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowExecutor.executeWorkflowStep(BaseWorkflowExecutor.java:681)
    at com.dtolabs.rundeck.core.execution.workflow.engine.StepCallable.apply(StepCallable.java:71)
    at com.dtolabs.rundeck.core.execution.workflow.engine.StepOperation.apply(StepOperation.java:76)
    at com.dtolabs.rundeck.core.execution.workflow.engine.StepOperation.apply(StepOperation.java:32)
    at com.dtolabs.rundeck.core.rules.WorkflowEngineOperationsProcessor.lambda$beginOperation$2(WorkflowEngineOperationsProcessor.java:330)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
    at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
    at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:29)
    at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:23)
    at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:43)
    at com.hierynomus.smbj.session.Session.connectTree(Session.java:110)
    ... 30 more
Caused by: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
    at com.hierynomus.protocol.commons.concurrent.PromiseBackedFuture.get(PromiseBackedFuture.java:60)
    at com.hierynomus.protocol.commons.concurrent.CancellableFuture.get(CancellableFuture.java:84)
    at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:38)
    ... 31 more
Caused by: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
    at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:27)
    at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:21)
    at com.hierynomus.protocol.commons.concurrent.Promise.deliverError(Promise.java:95)
    at com.hierynomus.smbj.connection.OutstandingRequests.handleError(OutstandingRequests.java:88)
    at com.hierynomus.smbj.connection.Connection.handleError(Connection.java:402)
    at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:53)
    ... 1 more
Caused by: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
    at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:53)
    at com.hierynomus.smbj.transport.PacketReader.readPacket(PacketReader.java:69)
    at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:47)
    ... 1 more
Caused by: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
    at com.hierynomus.protocol.commons.buffer.Buffer.ensureAvailable(Buffer.java:184)
    at com.hierynomus.protocol.commons.buffer.Buffer.skip(Buffer.java:732)
    at com.hierynomus.mssmb2.SMB2Error.read(SMB2Error.java:46)
    at com.hierynomus.mssmb2.SMB2Packet.readError(SMB2Packet.java:115)
    at com.hierynomus.mssmb2.SMB2Packet.read(SMB2Packet.java:109)
    at com.hierynomus.mssmb2.messages.SMB2MessageConverter.read(SMB2MessageConverter.java:79)
    at com.hierynomus.mssmb2.messages.SMB2MessageConverter.read(SMB2MessageConverter.java:42)
    at com.hierynomus.mssmb2.messages.SMB2MessageConverter.read(SMB2MessageConverter.java:85)
    at com.hierynomus.mssmb2.messages.SMB2MessageConverter.read(SMB2MessageConverter.java:26)
    at com.hierynomus.smbj.connection.Connection$DelegatingSMBMessageConverter.read(Connection.java:437)
    at com.hierynomus.smbj.connection.Connection$DelegatingSMBMessageConverter.read(Connection.java:426)
    at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readPacket(DirectTcpPacketReader.java:42)
    at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:49)
    ... 3 more
Failed: PluginFailed: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow
[workflow] finishExecuteNodeStep(sap-tst): NodeDispatch: PluginFailed: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: com.hierynomus.protocol.commons.buffer.Buffer$BufferException: Underflow

Running commands dispatched to the same node works without any problem:

[workflow] beginExecuteNodeStep(tst-node): NodeDispatch: StepExecutionItem{type='NodeDispatch', keepgoingOnSuccess=false, hasFailureHandler=false}
LocalExecNodeStepPlugin, running command (2): 'echo''JobID: ed67edd1-f187-4715-bc8b-3b2c811191cd'
JobID: ed67edd1-f187-4715-bc8b-3b2c811191cd
[workflow] finishExecuteNodeStep(tst-node): NodeDispatch: Step successful

After some deep-investgation with wireshark on the windows-destination i've found a problem: According to the logs rundeck/overthere-winrm is connection with SMBv2 using username and password: OK But then he is trying to reach a share on the win-node named "t$mp" (\[servername]\t$mp): image

There is no share named like that in standard configuration of windows. After creating a share named "t$mp" somewhere on the windows node everthing is working fine.

I am not able to create a "t$mp" share in all affected windows nodes in the future - where is that share-name defined and where can i configure that?

Node definiton:

tst-node:
  hostname: "WINSRV"
  username: [username]
  # os
  osName: "Windows"
  osFamily: "Windows"
  osVersion: "4.19.128-microsoft-standard"
  osArch: "amd64"
  # executor
  node-executor: "overthere-winrm"
  winrm-password-storage-path: "keys/winusr"
  file-copier: "overthere-winrm"
  winrm-auth-type: "basic" 
  winrm-cmd: "PowerShell"
  winrm-protocol: "http"

rundeck-winrm-plugin: rundeck-winrm-plugin-1.3.8.jar Rundeck Community Version 3.4.3

pts-aut commented 2 years ago

Update: the issue only occurs when the destination path in the Copy-Step is set as relative-path (i.e. temp/blah/...) - when using absolute path (i.e. C:/tmp/blah/...) everything is working fine.