password vault

[SudoMike]

Pyinfra2 is missing a useful feature from Ansible, which is the password vault. Oftentimes, you need to use credentials and other secrets in your tasks. For example, you might need to write a config file on the remote host that includes a password. Ideally, task code could do something like this:

vault = pyinfra.get_vault()
files.template(src=whatever, dest=whatever, some_password=vault.get('password.myserver.elasticsearch')

Ideally the vault is specified on the command line, and the password can be in a file:

pyinfra --vault ~/.config/my_vault --vault_pw_file ~/.config/my_vault_pw ...

Also, pyinfra would need to securely transfer files over to the other system. The main problem with the way it does it in ssh.py:put_file is that it creates a temp file and writes the contents of the file (potentially with a secret embedded in the text), and after that it moves the file to its proper location and sets the owner and mode bits. Before it moves the file, it could be accessed on the remote machine.

This means that even if someone implements their own secure vault, they still can't securely transfer files to the target system.

[gchazot]

These are 2 features being requested here:

1. A way to securely read passwords for use in a deploy
2. A secure way to create/update a file on the target without exposing the content to other users in the process.

I think 2. is a bug rather than a feature request.

About 1., pyinfra deploys are Python. If you can access your vault with Python code, then the feature is already builtin.

There's a basic example in the documentation already.

[SudoMike]

Agreed that 2 is a bug.

On 1, IMO having a vault is a pretty core common feature for any nontrivial deployment situation, and it would be very suboptimal to leave it to every developer to implement their own scheme for it. There are all kinds of ways to do it, and it's a nontrivial task to get it right. To be clear, I'm not proposing that pyinfra have bridges to various third party vaults like Keepass, but that it has its own vault file format, and password entry (or get-password-from-file) scheme. If I remember correctly, in Ansible the vault is basically an encrypted YAML file and it gets loaded in at startup so "code" can reference the variables in the YAML file.

[nikaro]

If you can access your vault with Python code, then the feature is already builtin.

Not sure about that, the way pyinfra works may conflict with the usage of your secret manager. For example i'm trying to use Google Secrets Manager:

from google.cloud import secretmanager_v1

def _get_secret(name: str) -> str:
    secret_manager = secretmanager_v1.SecretManagerServiceClient()
    secret = secret_manager.access_secret_version(name=name).payload.data.decode("utf-8").strip()
    return secret

mysecret = _get_secret("projects/mygcpproject/secrets/mysecret/versions/latest")
print(mysecret)

This standalone snippet works fine, but when used in pyinfra code it fails:

google.api_core.exceptions.DeadlineExceeded: 504 Deadline Exceeded

Edit: started a discussion here