Add support for non-POSIX target environments in the SSH connector

[KuxaBeast]

Description

I tried using pyinfra to write a few operations to deploy my infrastructure consisting of MikroTik RouterBoard-based devices running RouterOS, but found out, that the SSH connector has a hardcoded sh -c wrapper function in its code for executing remote commands.

After some investigation I found the culprit is this: (api/connectors/ssh.py:282) command = make_unix_command(command, state=state, **command_kwargs)

I don't know if I am missing something and there is a way to execute remote commands without this wrapper (which can only support devices running POSIX shell), but it would be nice to take into consideration adding support for non-POSIX target environments or at least to have the option to turn this feature off.

To Reproduce

Try to execute a valid command over SSH on virtually any target which doesn't ship a POSIX-compliant shell.

Expected behavior

The command executes without any issue on the target device.

Meta

• pyinfra --support

  System: Linux
    Platform: Linux-5.14.14-arch1-1-x86_64-with-glibc2.33
    Release: 5.14.14-arch1-1
    Machine: x86_64
  pyinfra: v1.4.17
  Executable: /home/user/.local/bin/pyinfra
  Python: 3.9.7 (CPython, GCC 11.1.0)

• How was pyinfra installed (source/pip)? pipx, since that's the only way that worked for me on Arch Linux
• Output with -vv and --debug

  
  $ pyinfra -vv --debug yum.ospf exec -- '/file print'
  [pyinfra_cli.main] Deploy directory remains as cwd
  --> Loading config...
  --> Loading inventory...
  [pyinfra_cli.inventory] Creating fake inventory...

--> Connecting to hosts... [pyinfra.api.connectors.ssh] Connecting to: yum.ospf ({'allow_agent': True, 'look_for_keys': True, 'hostname': 'yum.ospf', 'timeout': 10}) [yum.ospf] Connected [pyinfra.api.state] Activating host: yum.ospf [pyinfra.api.operation] Adding operation, {'Server/Shell'}, opOrder=(0,), opHash=784a97bf1955d5f7a2b9dd6c1e371e17b73c42bc

--> Proposed changes: Ungrouped: [yum.ospf] Operations: 1 Commands: 1

--> Beginning operation run... --> Starting operation: Server/Shell (/file print) [pyinfra.api.operations] Starting operation Server/Shell on yum.ospf [pyinfra.api.connectors.ssh] Running command on yum.ospf: (pty=None) sh -c '/file print' [yum.ospf] >>> sh -c '/file print' [yum.ospf] syntax error (line 1 column 7) [pyinfra.api.connectors.ssh] Waiting for exit status... [pyinfra.api.connectors.ssh] Command exit status: 1 [yum.ospf] Error [pyinfra.api.state] Failing hosts: yum.ospf --> pyinfra error: No hosts remaining!

With the (api/connectors/ssh.py:282) line manually commented out:

$ pyinfra -vv --debug yum.ospf exec -- '/file print' [pyinfra_cli.main] Deploy directory remains as cwd --> Loading config... --> Loading inventory... [pyinfra_cli.inventory] Creating fake inventory...

--> Connecting to hosts... [pyinfra.api.connectors.ssh] Connecting to: yum.ospf ({'allow_agent': True, 'look_for_keys': True, 'hostname': 'yum.ospf', 'timeout': 10}) [yum.ospf] Connected [pyinfra.api.state] Activating host: yum.ospf [pyinfra.api.operation] Adding operation, {'Server/Shell'}, opOrder=(0,), opHash=784a97bf1955d5f7a2b9dd6c1e371e17b73c42bc

--> Proposed changes: Ungrouped: [yum.ospf] Operations: 1 Commands: 1

--> Beginning operation run... --> Starting operation: Server/Shell (/file print) [pyinfra.api.operations] Starting operation Server/Shell on yum.ospf [pyinfra.api.connectors.ssh] Running command on yum.ospf: (pty=None) /file print [yum.ospf] >>> /file print [yum.ospf] # NAME TYPE SIZE CREATION-TIME
[yum.ospf] 0 skins directory jan/01/1970 02:00:05 [yum.ospf] 1 auto-before-reset.b... backup 16.4KiB jan/01/1970 02:00:06 [yum.ospf] 2 pub directory jan/02/1970 05:23:38 ... (output trimmed) ... [yum.ospf] [pyinfra.api.connectors.ssh] Waiting for exit status... [pyinfra.api.connectors.ssh] Command exit status: 0 [yum.ospf] Success

--> Results: Ungrouped: [yum.ospf] Successful: 1 Errors: 0 Commands: 1/1

[KuxaBeast]

Now realized it should've probably been a feature request but, well.

[Fizzadar]

Hi @KuxaBeast - it is currently possible to set a shell, but not disable it. There's two options for this:

• shell_executable global argument passed into operations (https://docs.pyinfra.com/en/1.x/deploys.html#global-arguments)
• The SHELL config variable

What's the target system you're working on, out of interest?

[KuxaBeast]

What's the target system you're working on, out of interest?

It's RouterOS, a proprietary Linux-based software from Mikrotik, which supports many architectures and devices, mainly their RouterBoards. They have their own implementation of SSH server and scripting command interface.

[KuxaBeast]

@Fizzadar, what would be the best approach to implement/fix this feature? This should be actually an opt-in functionality in the connector, or completely done by the user manually IMHO. Or maybe make a separate function from server.shell (something like server.subprocess) - something that does not allow for shell expressions, just plainly sends the string command to the target?

• shell_executable global argument passed into operations

also, I don't think it's the best idea to assume that any possible shell_executable will have the, currently hardcoded, -c argument for executing expressions - I don't know what is the CLI for Windows' cmd, but if it's a variable, it could be set to anything else

[Fizzadar]

@KuxaBeast apologies for the delay here. Agreed there's a few issues with the current handling. Currently Windows has it's own make command function. The -c argument is pretty standard across most POSIX shell implementations but I agree this shouldn't be completely hardcoded. I would propose immediately:

• Allow setting shell_executable=None which removes entirely the wrapping of commands under a shell executable

This should be a relatively simple change to the make_unix_command.

And longer term:

• Making the -c configurable somehow, with -c being the default
• Better handling shell executable between Windows + Unix systems