How to update file with values from stdin?

[aodj]

I'm trying to work out how I might be able to update the value in a yaml file that is piped in. My use case is that I'm encrypting files using Ansible Vault. Here's a contrived example:

$ echo -n "ham and eggs" | ansible-vault encrypt_string --stdin-name=ham-and-eggs-encrypted
New Vault password:
Confirm New Vault password:
Reading plaintext input from stdin. (ctrl-d to end input)
ham-and-eggs-encrypted: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          64656436373464613765353536353135653062373861373732396461316136353333633766623639
          6234663364346163396133386133643461643462616662330a653663366536326462316563663561
          30633565353464626239633031623333333966366263376161623537373631383866386233633033
          6631323736386332620a653335663462383334373836303334326461613236333862383766666165
          3964

I want to then pipe this into yq to update the stored value for the relevant key (ham-and-eggs-encrypted in this case).

I was hoping I could do something like this:

$ echo -n "ham and eggs" | ansible-vault encrypt_string --stdin-name=ham-and-eggs-encrypted | sed -E 's/^[ ]{8}//' | yq -yi $1 encrypted_vars.yml
yq: -i/--in-place can only be used with filename arguments, not on standard input

... but as you can see I get the error yq: -i/--in-place can only be used with filename arguments, not on standard input.

Am I missing some way to do this? or is it simply not possible with pipes?

[aodj]

I've got something that kind of works:

$ echo -n "ham and eggs" | ansible-vault encrypt_string --stdin-name=ham-and-eggs-encrypted | sed -E 's/^[ ]{8}//' | yq -Y -s 'add' encrypted_vars.yml /dev/stdin

This outputs a correctly combined yaml to my terminal, but whenever I try to pipe it back to the source file I only get the newly encrypted variable, overwriting the others in the file. I would really love to simply use the -i option in yq though 😉

[kislyuk]

You're looking to give yq/jq two streams at once. This is not going to work, since there is only one stdin, and using -i with stdin is meaningless (since it's read-only, and you're already getting the edited output on stdout).

What you need here instead is an environment variable or string substitution:

export new_value=$(echo -n "ham and eggs" | ansible-vault encrypt_string --stdin-name=ham-and-eggs-encrypted)
yq -Yi '.["ham-and-eggs-encrypted"]=env.new_value' encrypted_vars.yml

[aodj]

That seems to work pretty well however I'm getting a nested keyval now, since env.new_value is already a yaml expression; the output from the ansible-vault encrypt_string is key: val:

ham-and-eggs-encrypted: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          64656436373464613765353536353135653062373861373732396461316136353333633766623639
          6234663364346163396133386133643461643462616662330a653663366536326462316563663561
          30633565353464626239633031623333333966366263376161623537373631383866386233633033
          6631323736386332620a653335663462383334373836303334326461613236333862383766666165
          3964

Following your suggestion I end up with the following:

ham-and-eggs-encrypted: !vault |-
  ham-and-eggs-encrypted: !vault |
          $ANSIBLE_VAULT;1.1;AES256
            64656436373464613765353536353135653062373861373732396461316136353333633766623639
            6234663364346163396133386133643461643462616662330a653663366536326462316563663561
            30633565353464626239633031623333333966366263376161623537373631383866386233633033
            6631323736386332620a653335663462383334373836303334326461613236333862383766666165
            3964

[kislyuk]

It sounds like you need this then:

export new_value=$(echo -n "ham and eggs" | ansible-vault encrypt_string --stdin-name=ham-and-eggs-encrypted | yq -Y '.["ham-and-eggs-encrypted"]')
yq -Yi '.["ham-and-eggs-encrypted"]=env.new_value' encrypted_vars.yml

Are there other parts to this document? From your example it seems like ansible-vault is producing the entire document that you need.