HCP Vault Secrets integration not recognizing service principal environment variables

[arrrgi]

Describe the bug

Applying templates which use HCP Vault Secrets throws an error even though these are set in the env key in Chezmoi's config on first apply when combined as chezmoi init arrrgi --apply --branch=feature/setup-scripts

HCP Vault Secrets templating works fine when init and apply are performed as separate steps, ie.

chezmoi init arrrgi --branch=feature/setup-scripts
chezmoi apply

To reproduce

Run chezmoi init arrrgi --apply --verbose and provide Vault Secrets HCP_CLIENT_ID, HCP_CLIENT_SECRET, Organization and Project values via the promptStringOnce functions in .chezmoi.yaml.tmpl

Expected behavior

The repo is cloned, the prompted values are stored in Chezmoi config, the template fetches secrets to replace templated values.

Output of command with the --verbose flag

$ chezmoi init arrrgi --apply --branch=feature/setup-scripts --verbose
diff --git a/.config/git b/.config/git
new file mode 40755
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
--- /dev/null
+++ b/.config/git
diff --git a/.config/git/config b/.config/git/config
new file mode 100644
index 0000000000000000000000000000000000000000..8ea2b9c02ac3ac94ba005a77489f2a2bae7bedbe
--- /dev/null
+++ b/.config/git/config
@@ -0,0 +1,2 @@
+[include]
+  path = ~/.config/git/standard
chezmoi: template: dot_config/git/standard.tmpl:4:18: executing "dot_config/git/standard.tmpl" at <hcpVaultSecret "sshSigningKeyPersonal">: error calling hcpVaultSecret: /home/<redacted>/.local/bin/vlt secrets get --plaintext --app-name chezmoi --project <redacted> --organization <redacted> sshSigningKeyPersonal: exit status 1
You are not currently logged in. Please try using the vlt login command to get started

Output of chezmoi doctor

```console $ chezmoi doctor RESULT CHECK MESSAGE ok version v2.42.0, commit 694977b904e888ba285aa0fd44617d1d59d89bc7, built at 2023-11-26T18:41:00Z, built by goreleaser ok latest-version v2.42.0 ok os-arch linux/amd64 (Ubuntu 22.04.3 LTS (Jammy Jellyfish)) ok uname Linux PF1QJ252 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux ok go-version go1.21.4 (gc) ok executable ~/.local/bin/chezmoi ok upgrade-method replace-executable ok config-file ~/.config/chezmoi/chezmoi.yaml, last modified 2023-11-28T22:59:31+10:00 ok source-dir ~/.local/share/chezmoi is a git working tree (clean) ok suspicious-entries no suspicious entries ok working-tree ~/.local/share/chezmoi is a git working tree (clean) ok dest-dir ~ is a directory ok umask 022 ok cd-command found /bin/bash ok cd-args /bin/bash info diff-command not set ok edit-command found /usr/bin/vi ok edit-args /usr/bin/vi ok git-command found /usr/bin/git, version 2.34.1 ok merge-command found /usr/bin/vimdiff ok shell-command found /bin/bash ok shell-args /bin/bash info age-command age not found in $PATH ok gpg-command found /usr/bin/gpg, version 2.2.27 info pinentry-command not set info 1password-command op not found in $PATH info bitwarden-command bw not found in $PATH info bitwarden-secrets-command bws not found in $PATH info dashlane-command dcli not found in $PATH info doppler-command doppler not found in $PATH info gopass-command gopass not found in $PATH info keepassxc-command keepassxc-cli not found in $PATH info keepassxc-db not set info keeper-command keeper not found in $PATH info lastpass-command lpass not found in $PATH info pass-command pass not found in $PATH info passhole-command ph not found in $PATH info rbw-command rbw not found in $PATH info vault-command vault not found in $PATH ok vlt-command found ~/.local/bin/vlt, version 1.0.0 info secret-command not set ```

Additional context

The resulting ~/.config/chezmoi/chezmoi.yaml file contains:

diff:
  pager: delta
env:
  HCP_CLIENT_ID: "<redacted>"
  HCP_CLIENT_SECRET: "<redacted>"
encryption: age
age:
  identity: "/home/<redacted>/.config/age/key.txt"
  recipient: "<redacted>"
hcpVaultSecrets:
  organizationId: "<redacted>"
  projectId: "<redacted>"
  applicationName: chezmoi
hooks:
  read-source-state:
    pre:
      command: ".local/share/chezmoi/home/.hooks/.install-hcp-vlt.sh"
data:
  target:
    darwin: false
    debian: false
    devcontainer: false
    ubuntu: false
    wsl: true
  function:
    hybrid: false
    personal: false
    work: true
  access:
    controlled: true
    restricted: false
  privacy:
    private: true
    public: false
  state:
    ephemeral: true
    persistent: false
  secrets:
    apikeys: true
    sshkeys: true
    storagekeys: false
  git:
    ado_org: "<redacted>"
    ado_username: "<redacted>"
    email: "<redacted>"
    name: "Rowan Gillson"

[twpayne]

Thanks for reporting this. The issue was that chezmoi init --apply was not setting the environment variables defined in the config file after re-reading the config file. #3372 should fix this. Would you be able to test it?

[arrrgi]

I've not installed from source before but it's not beyond me, I'll test as soon as I can and let you know.

Thanks for the quick turnaround though, I'll endeavour to do the same.

[twpayne]

You can download a compiled binary for your architecture in the Artifacts section at the bottom of this page: https://github.com/twpayne/chezmoi/actions/runs/7023347012?pr=3372

If your OS/arch combo isn't listed please tell me.

[arrrgi]

Confirmed working with both linux/amd64 and darwin/arm64 with this latest build snapshot.