"chezmoi update" prompts for password even though encrypted file has not changed

[lefth]

Describe the bug

Two machines were pretty much in sync. On one machine I added a directory with "chezmoi add" and "chezmoi cd" then git pushing. On the next machine I tried to get these files with "chezmoi update". I was prompted for the password.

To reproduce

1. Configure chezmoi to use gpg with symmetric encryption on two machines. (I used two WSL distros on the same machine.)
2. Add a file to chezmoi with encryption.
3. Push the repository to git and get both machines in sync.
4. On machine A, add an unencrypted file, for example chezmoi add ~/.bashrc and push the change to the git remote.
5. On machine B, run chezmoi update

Expected behavior

I expect to not be prompted when encrypted files haven't been changed. Perhaps this could be accomplished by (optionally) storing a checksum along with the encrypted data.

I suggest an option because in some circumstances that would be too much information leakage.

In any case, so much password prompting is not reasonable--I have several encrypted files, so I get prompted several times, even if only one file needs to be updated (and it's not encrypted).

Output of command with the --verbose flag

$ chezmoi update --verbose
Enter passphrase for key '/home/user/.ssh/id_rsa':
Already up to date.
gpg: AES256.CFB encrypted data
gpg: gcry_kdf_derive failed: Invalid data
gpg: encrypted with 1 passphrase
gpg: decryption failed: Bad session key
chezmoi: home/user/.secret: exit status 2

Output of chezmoi doctor

```console $ chezmoi doctor RESULT CHECK MESSAGE ok version v2.48.0, commit c758a1c57fb6084631736a7dda5ac0aaeeffa946, built at 2024-04-26T20:38:57Z, built by goreleaser ok latest-version v2.48.0 ok os-arch linux/amd64 (Ubuntu 24.04 LTS (Noble Numbat)) ok uname Linux LAPTOP-DMH3HPFI 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux ok go-version go1.22.2 (gc) ok executable /snap/chezmoi/876/chezmoi ok upgrade-method snap-refresh ok config-file ~/.config/chezmoi/chezmoi.toml, last modified 2024-05-02T01:57:03+08:00 ok source-dir ~/.local/share/chezmoi is a git working tree (clean) ok suspicious-entries no suspicious entries ok working-tree ~/.local/share/chezmoi is a git working tree (clean) ok dest-dir / is a directory ok umask 022 ok cd-command found /bin/zsh ok cd-args /bin/zsh info diff-command not set ok edit-command found /usr/bin/nvim ok edit-args /usr/bin/nvim ok git-command found /usr/bin/git, version 2.43.0 ok merge-command found /usr/bin/vimdiff ok shell-command found /bin/zsh ok shell-args /bin/zsh info age-command age not found in $PATH ok gpg-command found /usr/bin/gpg, version 2.4.4 info pinentry-command not set info 1password-command op not found in $PATH info bitwarden-command bw not found in $PATH info bitwarden-secrets-command bws not found in $PATH info dashlane-command dcli not found in $PATH info doppler-command doppler not found in $PATH info gopass-command gopass not found in $PATH info keepassxc-command keepassxc-cli not found in $PATH info keepassxc-db not set info keeper-command keeper not found in $PATH info lastpass-command lpass not found in $PATH info pass-command pass not found in $PATH info passhole-command ph not found in $PATH info rbw-command rbw not found in $PATH info vault-command vault not found in $PATH info vlt-command vlt not found in $PATH info secret-command not set ```

Additional context

Note: I set the destination to be / in the config file. I don't think that is relevant to the problem.

[halostatue]

This is not a bug. Your suggestion (storing a checksum/hash) is an interesting one, but it is not risk-free, either.

In general, with anything templated via a password manager or encrypted, chezmoi cannot know whether the file has changed until the values are unlocked and templated or decrypted. If you have two classes of changes, you may want to look at #2574 for suggestions on how to layer your configuration so that your default storage is free of encrypted data and you have a second repo that only has encrypted data.

Your suggestion is interesting, and it might be possible to add a checksum / digest field to the state database for encrypted data, but:

1. it will make every encryption step slower, because chezmoi would need to read the file and compute a pre-encryption digest before encrypting the file (possibly through a shell call).
2. it may incorrectly decide that something has not changed.

The pre-encryption digest needs to be used because encrypting the same file at two different times with the same password may not result in the same bytes on output (different salts, etc.).

[twpayne]

As @halostatue says, chezmoi has to retrieve the secrets from your password manager to know if the file's contents have changed. There is no way to avoid this.

In any case, so much password prompting is not reasonable--I have several encrypted files, so I get prompted several times, even if only one file needs to be updated (and it's not encrypted).

The solution here is to use a password manager that supports some kind of temporary persistent session so you don't have to enter your password every time.

As you're currently using GPG you might be able to adapt this technique from age.

However, this path leads to accidentally implementing your own password manager, which I wouldn't recommend. Better to use to an existing off-the-shelf solution, which is likely to be much more secure.

Another option is encrypt entire files and then use the --exclude=encrypted flag to skip encrypted files when running chezmoi apply, but this will you to know when your files containing secrets have changed.

[lefth]

@twpayne @halostatue Thanks for all the tips, I will try your suggestions!

As for knowing whether an encrypted file has changed, there is no need to decrypt the encrypted contents if checksums are stored. If the tips provided work well (secure, good UX, cross platform) there is no need to pursue this, but the strategy that jumps out is: during encryption dot_ssh/id_rsa.chezmoichecksum would be created containing three lines: "v1", a random salt, and the sha256sum of ~/.ssh/id_rsa plus the salt.

A checksum would indeed need to be performed to check whether the file is changed, but checksums are generally faster than decryption. (So extra computation would only be needed when the file is changed--in that case both checksumming and encryption and/or would be needed.)

Checksum collisions aren't currently a concern. The negative media coverage a few years ago about md5sum being broken was exaggerated (tampering with one file to force a checksum collision still isn't possible), and sha256sum is much stronger.

But like I said, I will experiment with the setups you suggested.

[halostatue]

As for knowing whether an encrypted file has changed, there is no need to decrypt the encrypted contents if checksums are stored. If the tips provided work well (secure, good UX, cross platform) there is no need to pursue this, but the strategy that jumps out is: during encryption dot_ssh/id_rsa.chezmoichecksum would be created containing three lines: "v1", a random salt, and the sha256sum of ~/.ssh/id_rsa plus the salt.

A checksum would indeed need to be performed to check whether the file is changed, but checksums are generally faster than decryption. (So extra computation would only be needed when the file is changed--in that case both checksumming and encryption and/or would be needed.)

If we were to pursue a digest approach on this, it would need to be stored in the state file, not side-by-side with the decrypted file or in the source tree. The difficulty would then be determining how the checksum is associated with the encrypted file in such a way as to (a) not interfere with the decryption process and (b) not require additional files.

I think that this is an interesting concept, but should only be pursued as a possible v3 concept as there would likely need to be changes in how chezmoi handles encryption/decryption to add chezmoi envelopes for such a digest. I don’t know if it’s something we would pursue in the end, because right now the encryption is (more or less) pass-through to the underlying libraries and/or binaries.

[rubiin]

I find this annoying as well. Perhaps a flag maybe chezmoi update --skip-encrypted would help for skipping encrypted file. For a single file, this is not a issue but 5-10 files with different secrets , it does become cumbersome

[twpayne]

I find this annoying as well. Perhaps a flag maybe chezmoi update --skip-encrypted would help for skipping encrypted file.

This already exists:

$ chezmoi update --exclude=encrypted

For a single file, this is not a issue but 5-10 files with different secrets , it does become cumbersome

The solution is to use a system that only requires you to authenticate once.

[twpayne]

Hopefully this is now resolved. Please re-open if needed.