Support for customer-provided keys in AzureRM backend

[DMKRCY]

Use-cases

We store our state in an Azure storage account. Previously, we had been using the Azure backend, but due to security requirements restricting us from storing secrets in an Azure storage account (due to the risk of an account being misconfigured as public) we have had to adopt a workaround where we manage the state file outside of terraform and use a local backend. Basically what we do is:

1. Download encrypted state from blob
2. Decrypt the state
3. Run terraform
4. Encrypt the state
5. Upload the state

This is obviously not an ideal solution

Proposal

Azure storage accounts provide the option to use a customer-provided encryption key by specifying it in the request https://docs.microsoft.com/en-us/azure/storage/blobs/encryption-customer-provided-keys

Even if the storage account is public, any subsequent request to read or modify the blob must include the encryption key, or it will be rejected.

The proposal is to introduce support for this by adding 3 new configuration options within the azurerm backend: encryption_key, encryption_key_sha256, and encryption_algorithm.

These options would then be passed into the giovanniBlobClient via the options struct for each of the relevant requests (i.e get blob, put blob, set blob metadata, get/set blob properties). This will require updating github.com/tombuildsstuff/giovanni as well to support the 3 required headers (x-ms-encryption-key, x-ms-encryption-key-sha256, x-ms-encryption-algorithm).

I've thrown together a quick POC to show what the changes could look like.

https://github.com/tombuildsstuff/giovanni/compare/master...DMKRCY:cmksupportbackport?expand=1 https://github.com/hashicorp/terraform/compare/main...DMKRCY:cmksupport?expand=1

I'm more than happy to complete the implementation of this myself and open a PR, but as per the contributing guidelines I want to get feedback and approval before investing any more time in this. The changes seem pretty simple to me but I could be missing something.

Let me know if there are any questions.

Thank you!

[crw]

Thanks for the submission. Just for your reference, there are a variety of other open issues and PRs around state encryption. See issues https://github.com/hashicorp/terraform/issues/9556, https://github.com/hashicorp/terraform/issues/29272 and PRs https://github.com/hashicorp/terraform/pull/28603, https://github.com/hashicorp/terraform/pull/29289. Just an FYI and for future viewers of this ticket. Thanks again for your submission!

[archmangler]

Has there been any progress on this? Basically CMK encryption for terraform backend state is mandated for enterprise security compliance in many enterprises these days. We have one such case.

If there is any workaround that would allow terraform to decrypt state files stored in a CMK-encrypted azure storage account that would be useful as well!

Currently I'm getting this:

+ terraform fmt
+ terraform init
Error loading state: blobs.Client#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="KeyVaultEncryptionKeyNotFound" Message="The key vault key is not found to unwrap the encryption key.\nRequestId:6ed9e67a-401e-0051-7a28-320ead000000\nTime:2023-01-27T08:19:48.7696249Z"

With this backend config:

  backend "azurerm" {
    resource_group_name  = "lol-caf-ea-nprd-rg-001"
    storage_account_name = "lolcafeanprdst001"
    container_name       = "lol-caf-ea-nprd-stct-001"
    key                  = "lol-azure-caf-dnsrecords-subnets.tfstate"
  }

[aharo-lumificyber]

The same in here

[fschmied]

@crw Any chance that #29289 can be merged to remedy that the azurerm backend cannot encrypt its state files? Or #28603, for all providers?

[crw]

@fschmied Thanks for re-raising https://github.com/hashicorp/terraform/pull/29289. Although I cannot comment on the chances of these being accepted, I do raise them for reconsideration as circumstances change.

[hkrutzer]

@crw can you provide some insight into the factors contributing to the duration of the review process of these proposed changes?

[crw]

@hkrutzer We try to relay reasoning in the specific PRs when possible. It would be best to ask on the PR in question. It is difficult to give a succinct answer to this question, and I would like to keep this thread on-topic to the issue at-hand.

[riemers]

I bumped into this issue, but i just had incorrect rights. In the end it just worked, even with CMK storage turned on and using an HSM. As long as your client has rights to do the decryption of the key, it should work.

In particular i had the Status=403 Code="KeyVaultEncryptionKeyNotFound" Message="The key vault key is not found to unwrap the encryption key error message. Also if you set a role assignment it does take some time before it comes into effect hence why i found this github issue only to find out it started to work again after adding the role assignment. I could be wrong just mentioning my findings.