How to run long running script via SSH?

[bab5470]

I am trying to run a script which sets permissions on a large filesystem. It can take up to 2 hours to complete.

I am trying to use the following powershell code:

$session = New-SSHSession -ComputerName $server -Credential $credential -Force -WarningAction SilentlyContinue -ConnectionTimeout 3600 
$result = Invoke-SSHCommand -Command $command -SSHSession $session 

In this case, $command=/scripts/perms.sh which is a shell script with a recursive chmod and chown command in them.

As you can see I have set the ConnectionTimeout to 3600 (60 minutes) however when I run this it returns within a few seconds, even if the script hasn't finished running.

Exception             : System.Management.Automation.MethodInvocationException: Exception calling "EndExecute" with
                        "1" argument(s): "Command 'sudo /scripts/perms.sh' has timed out." --->
                        Renci.SshNet.Common.SshOperationTimeoutException: Command 'sudo
                        /ptp/scripts/wordpress-perms.sh' has timed out.
                           at Renci.SshNet.SshCommand.WaitOnHandle(WaitHandle waitHandle)
                           at Renci.SshNet.SshCommand.EndExecute(IAsyncResult asyncResult)
                           at CallSite.Target(Closure , CallSite , Object , Object )
                           --- End of inner exception stack trace ---
                           at
                        System.Management.Automation.ExceptionHandlingOps.ConvertToMethodInvocationException(Exception
                        exception, Type typeToThrow, String methodName, Int32 numArgs, MemberInfo memberInfo)
                           at CallSite.Target(Closure , CallSite , Object , Object )
                           at <ScriptBlock>(Closure , FunctionContext )
TargetObject          :
CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
FullyQualifiedErrorId : SshOperationTimeoutException
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1:
                        line 294
                        at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1:
                        line 264
                        at CheckAsyncProcessing<Process>, C:\Program
                        Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1: line 260
                        at Invoke-SSHCommand<End>, C:\Program
                        Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1: line 399
                        at Run-SSH-Commands, C:\ptp\scripts\includes\run-ssh-comands.ps1: line 28
                        at SprintPrepContent, C:\ptp\scripts\SprintPrepSyncWordpressContent.ps1: line 165
                        at <ScriptBlock>, C:\ptp\scripts\SprintPrepSyncWordpressContent.ps1: line 189
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}
PSMessageDetails      :

Is there a way to set a higher timeout or keep the connection alive for longer?

[darkoperator]

wow that is one unique case. Only things that comes to mind is to run your script with nohup:

nohup your_script.sh &

This will create a nohup.out file with the script's output. The script will continue running after you log out

[bab5470]

Perfect - I should have thought of that - thank you. Let me give it a try and I'll let you know how that goes.

[bab5470]

Sadly that didn't work:

Exception             : System.Management.Automation.MethodInvocationException: Exception calling "EndExecute" with
                        "1" argument(s): "Command 'nohup sudo /scripts/perms.sh &' has timed out." --->
                        Renci.SshNet.Common.SshOperationTimeoutException: Command 'nohup sudo
                        /scripts/perms.sh &' has timed out.
                           at Renci.SshNet.SshCommand.WaitOnHandle(WaitHandle waitHandle)
                           at Renci.SshNet.SshCommand.EndExecute(IAsyncResult asyncResult)
                           at CallSite.Target(Closure , CallSite , Object , Object )
                           --- End of inner exception stack trace ---
                           at
                        System.Management.Automation.ExceptionHandlingOps.ConvertToMethodInvocationException(Exception
                        exception, Type typeToThrow, String methodName, Int32 numArgs, MemberInfo memberInfo)
                           at CallSite.Target(Closure , CallSite , Object , Object )
                           at <ScriptBlock>(Closure , FunctionContext )
TargetObject          :
CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
FullyQualifiedErrorId : SshOperationTimeoutException
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1:
                        line 294
                        at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1:
                        line 264
                        at CheckAsyncProcessing<Process>, C:\Program
                        Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1: line 260
                        at Invoke-SSHCommand<End>, C:\Program
                        Files\WindowsPowerShell\Modules\Posh-SSH\3.2.0\Posh-SSH.psm1: line 399
                        at Run-SSH-Commands, C:\ptp\scripts\includes\run-ssh-comands.ps1: line 28
                        at SprintPrepContent, C:\ptp\scripts\SprintPrepSyncWordpressContent.ps1: line 165
                        at <ScriptBlock>, C:\ptp\scripts\SprintPrepSyncWordpressContent.ps1: line 189
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}
PSMessageDetails      :

[bab5470]

Never mind - user error on my part. The correct command for me was:

nohup sudo /scripts/perms.sh > /dev/null 2>&1 &

Thank you @darkoperator !