SSH CopyFileFailed for recurisve copy after update to rundeck 5.3.0/5.4.0

[pts-aut]

With rundeck version 5.3.0 and 5.4.0 the FileCopier for SSHJ is not working for recursive copy. (With Version 5.1.2 and 5.2.0 everything is OK)

Error:

Begin copy 4096 bytes to node xxxxx@yyyy: /home/rundeck/scripts/linux -> /tmp/1133100-xxxx@yyyy/linux/
failed: Configuration error: null
Failed: CopyFileFailed: com.dtolabs.rundeck.core.execution.service.FileCopierException: Configuration error: null

Defintion:

sequence:
    commands:
    - exec: mkdir -p /tmp/${job.execid}-${node.name}/
    - configuration:
        destinationPath: /tmp/${job.execid}-${node.name}/linux/
        echo: 'true'
        recursive: 'true'
        sourcePath: /home/rundeck/scripts/linux/
      nodeStep: true
      type: copyfile

Default-File-Copier:

service.FileCopier.default.provider=sshj-scp
service.NodeExecutor.default.provider=sshj-ssh

• Rundeck version: 5.3.0 and 5.4.0 (With Version 5.1.2 everything is OK)
• install type: docker
• DB Type/version: postgres

Working with:

rundeck: 
  apiversion: 47
  base: /home/rundeck
  build: 5.2.0-20240410
  buildGit: v5.2.0-0-g1fda5a1
  node: rundeck
  serverUUID: a14bc3e6-75e8-4fe4-a90d-a16dcc976bf6
  version: 5.2.0-20240410

Error with:

rundeck: 
  apiversion: 47
  base: /home/rundeck
  build: 5.3.0-20240520
  buildGit: v5.3.0-0-g20f75b8
  node: rundeck
  serverUUID: a14bc3e6-75e8-4fe4-a90d-a16dcc976bf6
  version: 5.3.0-20240520

and

rundeck: 
  apiversion: 48
  base: /home/rundeck
  build: 5.4.0-20240618
  buildGit: v5.4.0-0-gb356f0f
  node: rundeck
  serverUUID: a14bc3e6-75e8-4fe4-a90d-a16dcc976bf6
  version: 5.4.0-20240618

Expected behavior Clean copy as it works in rundeck 5.2.0:

Begin copy 4096 bytes to node xxxxx@yyyy: /home/rundeck/scripts/linux -> /tmp/1135795-xxxxx@yyyy/linux/
Copied: /tmp/automatics/xxxxx@yyyy/linux/

[MegaDrive68k]

Confirmed with the following job definition:

- defaultTab: nodes
 description: ''
 executionEnabled: true
 id: 4632749f-12e3-4f0a-b2f7-f9451cff3baf
 loglevel: INFO
 name: CopyFiles
 nodeFilterEditable: false
 nodefilters:
   dispatch:
     excludePrecedence: true
     keepgoing: false
     rankOrder: ascending
     successOnEmptyNodeFilter: false
     threadcount: '1'
   filter: 'name: node00 '
 nodesSelectedByDefault: true
 plugins:
   ExecutionLifecycle: {}
 scheduleEnabled: true
 sequence:
   commands:
   - configuration:
       destinationPath: /home/vagrant/
       echo: 'true'
       recursive: 'true'
       sourcePath: /home/user/Downloads/origin/
     nodeStep: true
     type: copyfile
   keepgoing: false
   strategy: node-first
 uuid: 4632749f-12e3-4f0a-b2f7-f9451cff3baf

It works on Rundeck 5.2. It fails on 5.3/5.4 with the following output:

[2024-08-05T18:16:47,529] ERROR node.NodeStepPluginAdapter [pool-21-thread-1] - Error executing node step.
com.dtolabs.rundeck.core.execution.workflow.steps.node.NodeStepException: com.dtolabs.rundeck.core.execution.service.FileCopierException: Configuration error: null
   at com.dtolabs.rundeck.plugin.copyfile.CopyFileNodeStepPlugin.copyFile(CopyFileNodeStepPlugin.java:177) ~[?:?]
   at com.dtolabs.rundeck.plugin.copyfile.CopyFileNodeStepPlugin.executeNodeStep(CopyFileNodeStepPlugin.java:92) ~[?:?]
   at com.dtolabs.rundeck.core.execution.workflow.steps.node.NodeStepPluginAdapter.executeNodeStep(NodeStepPluginAdapter.java:169) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeNodeStep(ExecutionServiceImpl.java:207) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:130) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.dispatch.SequentialNodeDispatcher.dispatch(SequentialNodeDispatcher.java:61) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.dispatchToNodesWith(ExecutionServiceImpl.java:263) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.dispatchToNodes(ExecutionServiceImpl.java:234) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.workflow.steps.NodeDispatchStepExecutor.executeWorkflowStep(NodeDispatchStepExecutor.java:66) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeStep(ExecutionServiceImpl.java:111) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowExecutor.executeWFItem(BaseWorkflowExecutor.java:285) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowExecutor.executeWorkflowStep(BaseWorkflowExecutor.java:681) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.workflow.engine.StepCallable.apply(StepCallable.java:71) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.workflow.engine.StepOperation.apply(StepOperation.java:76) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.execution.workflow.engine.StepOperation.apply(StepOperation.java:32) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.core.rules.WorkflowEngineOperationsProcessor.lambda$beginOperation$1(WorkflowEngineOperationsProcessor.java:323) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131) [guava-32.0.1-jre.jar!/:?]
   at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75) [guava-32.0.1-jre.jar!/:?]
   at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82) [guava-32.0.1-jre.jar!/:?]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
   at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: com.dtolabs.rundeck.core.execution.service.FileCopierException: Configuration error: null
   at com.plugin.sshjplugin.SSHJFileCopierPlugin.copyFile(SSHJFileCopierPlugin.java:175) ~[?:?]
   at com.plugin.sshjplugin.SSHJFileCopierPlugin.copyFile(SSHJFileCopierPlugin.java:103) ~[?:?]
   at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.fileCopyFile(ExecutionServiceImpl.java:320) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.plugin.copyfile.CopyFileNodeStepPlugin.copyFile(CopyFileNodeStepPlugin.java:166) ~[?:?]
   ... 21 more
Caused by: java.lang.NullPointerException
   at java.io.File.<init>(File.java:278) ~[?:?]
   at net.schmizz.sshj.xfer.FileSystemFile.<init>(FileSystemFile.java:34) ~[?:?]
   at com.plugin.sshjplugin.model.SSHJScp.execute(SSHJScp.java:88) ~[?:?]
   at com.plugin.sshjplugin.SSHJFileCopierPlugin.copyFile(SSHJFileCopierPlugin.java:173) ~[?:?]
   at com.plugin.sshjplugin.SSHJFileCopierPlugin.copyFile(SSHJFileCopierPlugin.java:103) ~[?:?]
   at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.fileCopyFile(ExecutionServiceImpl.java:320) ~[rundeck-core-5.4.0-20240618.jar!/:?]
   at com.dtolabs.rundeck.plugin.copyfile.CopyFileNodeStepPlugin.copyFile(CopyFileNodeStepPlugin.java:166) ~[?:?]
   ... 21 more

Thanks for your feedback @pts-aut.

[pts-aut]

@MegaDrive68k Is there a new status on this problem? Is a fix announced for a next release?

[pts-aut]

Issue still present in Version 5.7.0

[wasimb-atica]

Even I am facing issue with Rundeck community Version 5.7.0