Ability to use sudo midway through script execution

[taranlu-houzz]

Is your feature request related to a problem? Please describe. I'm currently trying to use pyinfra to install brew. The target machine that I am working on is running Windows 10 with WSL. I have ssh enabled, and am able to issue pyinfra commands that connect to WSL and work (mostly) correctly.

The standard way to install brew is to use: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)". To avoid the prompt in the script, I am using the env var env CI=1, however I am running into a snag with the sudo prompt that comes up midway through the script. I cannot just run the script as root or using sudo, because it explicitly checks for this and disallows it, but I still need to be able to pass a sudo prompt midway through for it to install brew at the standard location in /home/linuxbrew/.linuxbrew. I am wondering if there is way to get around this?

Based on the discussion in #305, I thought I might be able to get the password from the user in the deploy using getpass, and then feed that to the brew install script using the stdin global parameter, but that didn't seem to work.

If I cannot find a solution I could write my own installation script, but I was hoping to be able to use the standard installation method that already exists.

Describe the solution you'd like I'm not entirely sure what the ideal solution would look like for this, since it is a pretty specific case... If the stdin solution I had mentioned worked, then maybe it would be nice to be able to "hold onto" the password that is captured via use_sudo_password so that it can be used later? It may also be the case that there is a better supported way to automate installation of brew, but I haven't had any luck finding an easy method.

[taranlu-houzz]

Would it maybe be possible to make it so that if you use use_sudo_password=True but do not use sudo=True it prompts for the sudo password and stores that somewhere, but then tries to run the command using the non-root user and only elevates using sudo when that is triggered (via SUDO_ASKPASS or something like that?).

[Fizzadar]

Hi @taranlu-houzz! So the issue here is detecting when the password is needed. The stdin command just pipes it into the command direct and so won't work where the password is needed down the line. Not currently sure on how to handle this situation!

[taranlu-houzz]

Hi @Fizzadar, based on what I was reading from the conversation in #305, I was wondering if it would be possible to grab the sudo password and setup the env var and askpass, but still only run the command as a regular user if the sudo=True arg is not given? That way, when it hits the sudo prompt, the SUDO_ASKPASS would kick in, right? Or is the not how it works?

[Fizzadar]

So this does work - but sudo needs to be called with -A, and brew appears to handle this nicely!

I'll make the change to enable use_sudo_password to work irrespective of whether sudo is set.

[Fizzadar]

This is now supported as of https://github.com/Fizzadar/pyinfra/commit/ff40f50f840c9a73a041aa0d8d80defd2375f6ef; expecting to have a dev version of 1.3 out today/tomorrow.

[taranlu-houzz]

Awesome! I'll try to test that out soon.

[taranlu-houzz]

Hmmm, installed the latest master branch using pipx and am now getting a different error. Looks like it is leaving out "sudo" when it tries to make the sudo call: [ws] Failed during: -A /bin/mkdir -p /home/linuxbrew/.linuxbrew

❯ pyinfra ws deploys/tasks/base/install_brew.py --debug
    [pyinfra_cli.main] Checking potential directory: deploys/tasks/base
    [pyinfra_cli.main] Setting directory to: deploys/tasks/base
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    [pyinfra_cli.inventory] Looking for group data in: deploys/tasks/base/group_data/----.py
    [pyinfra_cli.inventory] Looking for group data in: deploys/tasks/base/group_data/----.py

--> Connecting to hosts...
    [pyinfra.api.connectors.ssh] Connecting to: ws ({'allow_agent': True, 'look_for_keys': True, 'hostname': 'ws', 'timeout': 10})
    [ws] Connected
    [pyinfra.api.state] Activating host: ws

--> Preparing operations...
    Loading: deploys/tasks/base/install_brew.py
    [pyinfra.local] Including local file: /----/deploys/facts/hz_brew.py
    [pyinfra.api.state] Starting deploy /----/deploys/facts/hz_brew.py (args={}, data=None)
    [pyinfra.api.state] Reset deploy to None (args=None, data=None)
    [pyinfra.api.facts] Getting fact: hz_brew_exec_path (ensure_hosts: (Host(ws),))
    [pyinfra.api.connectors.ssh] Running command on ws: (pty=False) sh -c '
out_path="__EMPTY_STRING__"

# Check user specified path
if [ "${out_path}" = "__EMPTY_STRING__" ]; then
    if [ "__EMPTY_STRING__" != "__EMPTY_STRING__" ]; then
        [ -f "__EMPTY_STRING__" ] && out_path="__EMPTY_STRING__"
    fi
fi

# Check default macOS brew location
if [ "${out_path}" = "__EMPTY_STRING__" ]; then
    [ -f "/usr/local/bin/brew" ] && out_path="/usr/local/bin/brew"
fi

# Check default Linux brew location
if [ "${out_path}" = "__EMPTY_STRING__" ]; then
    [ -f "/home/linuxbrew/.linuxbrew/bin/brew" ] &&
        out_path="/home/linuxbrew/.linuxbrew/bin/brew"
fi

echo "${out_path}"
'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    [pyinfra.api.facts] Loaded fact hz_brew_exec_path
    [pyinfra.api.operation] Adding operation, {'Install brew via installer script.'}, opOrder=(42,), opHash=b9dbb542df63146caa6bd4fe1d4f57d808134f7a
    [ws] Ready: deploys/tasks/base/install_brew.py

--> Proposed changes:
    Ungrouped:
    [ws]   Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Install brew via installer script.
    [pyinfra.api.operations] Starting operation Install brew via installer script. on ws
    [pyinfra.api.connectors.ssh] Running command on ws: (pty=False) sh -c 'chmod +x pyinfra-sudo-askpass'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
[ws] sudo password:
    [pyinfra.api.connectors.ssh] Running command on ws: (pty=None) env SUDO_ASKPASS=pyinfra-sudo-askpass *** sh -c 'env CI=1 /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)\"'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 1
    [ws] ==> This script will install:
    [ws] /home/linuxbrew/.linuxbrew/bin/brew
    [ws] /home/linuxbrew/.linuxbrew/share/doc/homebrew
    [ws] /home/linuxbrew/.linuxbrew/share/man/man1/brew.1
    [ws] /home/linuxbrew/.linuxbrew/share/zsh/site-functions/_brew
    [ws] /home/linuxbrew/.linuxbrew/etc/bash_completion.d/brew
    [ws] /home/linuxbrew/.linuxbrew/Homebrew
    [ws] ==> The following new directories will be created:
    [ws] /home/linuxbrew/.linuxbrew/bin
    [ws] /home/linuxbrew/.linuxbrew/etc
    [ws] /home/linuxbrew/.linuxbrew/include
    [ws] /home/linuxbrew/.linuxbrew/lib
    [ws] /home/linuxbrew/.linuxbrew/sbin
    [ws] /home/linuxbrew/.linuxbrew/share
    [ws] /home/linuxbrew/.linuxbrew/var
    [ws] /home/linuxbrew/.linuxbrew/opt
    [ws] /home/linuxbrew/.linuxbrew/share/zsh
    [ws] /home/linuxbrew/.linuxbrew/share/zsh/site-functions
    [ws] /home/linuxbrew/.linuxbrew/var/homebrew
    [ws] /home/linuxbrew/.linuxbrew/var/homebrew/linked
    [ws] /home/linuxbrew/.linuxbrew/Cellar
    [ws] /home/linuxbrew/.linuxbrew/Caskroom
    [ws] /home/linuxbrew/.linuxbrew/Homebrew
    [ws] /home/linuxbrew/.linuxbrew/Frameworks
    [ws] ==> -A /bin/mkdir -p /home/linuxbrew/.linuxbrew
    [ws] environment: line 112: -A: command not found
    [ws] Failed during: -A /bin/mkdir -p /home/linuxbrew/.linuxbrew
    [ws] Error
    [pyinfra.api.state] Failing hosts: ws
--> pyinfra error: No hosts remaining!

❯ pipx list
venvs are in /Users/----/.local/pipx/venvs
apps are exposed on your $PATH at /Users/----/.local/bin
   package autoflake 1.4, Python 3.7.8
    - autoflake
   package black 20.8b1, Python 3.7.8
    - black
    - black-primer
    - blackd
   package coverage 5.3, Python 3.7.8
    - coverage
    - coverage-3.7
    - coverage3
   package doq 0.6.3, Python 3.7.8
    - doq
   package flake8 3.8.4, Python 3.7.8
    - flake8
   package ipython 7.19.0, Python 3.7.8
    - iptest
    - iptest3
    - ipython
    - ipython3
   package isort 5.6.4, Python 3.7.8
    - isort
   package jupyterlab 2.2.9, Python 3.7.8
    - jlpm
    - jupyter-lab
    - jupyter-labextension
    - jupyter-labhub
   package pyinfra 1.3.dev0, Python 3.7.8
    - pyinfra
   package pytest 6.1.2, Python 3.7.8
    - py.test
    - pytest

❯ pyinfra --version
pyinfra: v1.3.dev0

[Fizzadar]

Unfortunately that's a bug in brew/linuxbrew by the looks of it - pyinfra cannot change what's executing there. You can verify by running the command by hand:

First edit pyinfra-sudo-askpass and make it echo the sudo password directly, then run:

env SUDO_ASKPASS=pyinfra-sudo-askpass sh -c 'env CI=1 /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)\"'

Should give the same result direct which can be passed onto the homebrew team!

[Fizzadar]

This is now supported by https://github.com/Fizzadar/pyinfra/commit/ff40f50f840c9a73a041aa0d8d80defd2375f6ef & released in v1.3!

[beeblebrox3]

@taranlu-houzz dit it worked? I'm trying to do the same thing, but isnt working for me.

[beeblebrox3]

For the record: i did it. With a combination of _sudo=True, _sudo_user and _preserve_sudo_env.