When using the ansible_vault resource to decrypt a file, I believe the provider is storing the values of vault_password_file and vault_file in the Terraform state file. This causes a problem if Terraform is run from a host with a different location for those files. Examples where this could happen include two developers with different locations for these files, or a developer running locally and an automated pipeline (e.g. Gitlab Runner).
The problem is that the ansible_vault resource stores data in the state file including the absolute path to each file, which may be different when the resource is inspected at a later time.
In my case I am using Terragrunt as a wrapper around Terraform to enable different absolute locations for the password file and the vault file, specifically to support use between local development and in a Gitlab Runner. Terraform provides the function pathexpand() to resolve ~, and Terragrunt adds the function get_repo_root() to find the root of the current git repo. I am using those functions to make the Terrgrunt/Terrform config portable between developers and developer vs Runner configurations.
The problem arose when I ran terraform apply (via terragrunt apply) locally on a developer machine.
[excerpt from terragrunt.hcl to show how the values are dynamically computed]
the_vault_password_file=pathexpand("~/.config/my-vault-password") => /home/myuser/.config/my-vault-password
the_vault="${get_repo_root()}/my-vault" => /home/myuser/project1/my-vault.yml
These are passed to Terraform (the mechanism is unimportant here but uses a vars.tf file with these variables):
When this same repo is used in Gitlab Runner, these paths are different locally on the Runner (using the docker executor):
vault_password_file is located locally at ```/root/.config/my-vault-password
vault_file is located locally at /builds/project1/my-vault.yml
It is not an important detail exactly what these paths are, only that they differ from the values stored in the state file.
When the ansible_vault resource is compared to the state file, the previously stored values for vault_password_file and vault_file are used, which point to locations that do not exist in the current execution environment.
The following error is produced:
[WARNING]: Error getting vault password file (default): The vault password file /home/myuser/.config/my-vault-password was not found
ERROR! Unable to read source file (/home/myuser/project1/my_vault.yml): [Errno 2] No such file or directory: '/home/myuser/project1/my-vault.yml'
with ansible_vault.vault, on main.tf line 6, in resource "ansible_vault" "vault":
6: resource "ansible_vault" "vault" {
IMPORTANT The reason this error occurs is that the paths referenced in the error do not exist in the execution environment because they are in different absolute locations.
Possible solution
A possible solution is to not store the paths to vault_password_file and vault_file in the Terraform state file. I am not clear why they are stored in the first place, so perhaps they can be eliminated. At any rate, storing paths to resources whose locations may change is fragile.
Alternative
There are others ways to solve this problem that do not use this provider at all and pass in the decrypted value into Terraform. I was trying to use this provider to make the job easier.
When using the
ansible_vaultresource to decrypt a file, I believe the provider is storing the values ofvault_password_fileandvault_filein the Terraform state file. This causes a problem if Terraform is run from a host with a different location for those files. Examples where this could happen include two developers with different locations for these files, or a developer running locally and an automated pipeline (e.g. Gitlab Runner).The problem is that the
ansible_vaultresource stores data in the state file including the absolute path to each file, which may be different when the resource is inspected at a later time.In my case I am using Terragrunt as a wrapper around Terraform to enable different absolute locations for the password file and the vault file, specifically to support use between local development and in a Gitlab Runner. Terraform provides the function
pathexpand()to resolve~, and Terragrunt adds the functionget_repo_root()to find the root of the current git repo. I am using those functions to make the Terrgrunt/Terrform config portable between developers and developer vs Runner configurations.The problem arose when I ran
terraform apply(viaterragrunt apply) locally on a developer machine.These are passed to Terraform (the mechanism is unimportant here but uses a
vars.tffile with these variables):After
`terraform apply, the paths in the state file reference my local config (assuming this is how they are stored):When this same repo is used in Gitlab Runner, these paths are different locally on the Runner (using the
dockerexecutor):vault_password_fileis located locally at ```/root/.config/my-vault-passwordvault_fileis located locally at/builds/project1/my-vault.ymlIt is not an important detail exactly what these paths are, only that they differ from the values stored in the state file.
When the
ansible_vaultresource is compared to the state file, the previously stored values forvault_password_fileandvault_fileare used, which point to locations that do not exist in the current execution environment.The following error is produced:
IMPORTANT The reason this error occurs is that the paths referenced in the error do not exist in the execution environment because they are in different absolute locations.
Possible solution
A possible solution is to not store the paths to
vault_password_fileandvault_filein the Terraform state file. I am not clear why they are stored in the first place, so perhaps they can be eliminated. At any rate, storing paths to resources whose locations may change is fragile.Alternative
There are others ways to solve this problem that do not use this provider at all and pass in the decrypted value into Terraform. I was trying to use this provider to make the job easier.