Closed EZhao-Quest closed 1 year ago
HI @EZhao-Quest i have tried with you config and found that you can get the plain_text_value in fact if I understand you correct.
@wuxu92 yes, use nonsenstive can get th data. but there is a case, my test string include space, like this "this is a test", the result missing space, return the value "thisisatsg".
I also tried convert "this is a test" to base64 string, then get encrypted_data, put the encrypted_data to azurerm_key_vault_encrypted_value.decrypted block. the output missing padding, like this "dGhpcyBpcyBhIHRlc3Q", the expected value should be "dGhpcyBpcyBhIHRlc3Q="
I am also affected by this issue, to me it looks like a padding problem somewhere, see this minimal example:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">=3.45.0"
}
}
}
provider "azurerm" {
features {}
}
data "azurerm_client_config" "current" {
}
resource azurerm_resource_group "test" {
name = "test"
location = "westeurope"
}
resource "random_string" "random" {
length = 8
special = false
upper = false
}
resource "azurerm_key_vault" "test" {
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
name = "test-${random_string.random.id}"
sku_name = "standard"
tenant_id = data.azurerm_client_config.current.tenant_id
}
resource "azurerm_key_vault_key" "test" {
key_opts = [
"decrypt",
"encrypt",
"sign",
"unwrapKey",
"verify",
"wrapKey",
]
key_type = "RSA"
key_vault_id = azurerm_key_vault.test.id
name = "test"
key_size = 2048
}
data "azurerm_key_vault_encrypted_value" "encrypt" {
key_vault_key_id = azurerm_key_vault_key.test.id
algorithm = "RSA1_5"
plain_text_value = "some-secret"
}
data "azurerm_key_vault_encrypted_value" "decrypt" {
key_vault_key_id = azurerm_key_vault_key.test.id
algorithm = "RSA1_5"
encrypted_data = data.azurerm_key_vault_encrypted_value.encrypt.encrypted_data
}
output "encrypted" {
value = nonsensitive(data.azurerm_key_vault_encrypted_value.encrypt.encrypted_data)
}
output "decrypted" {
value = nonsensitive(data.azurerm_key_vault_encrypted_value.decrypt.plain_text_value)
}
the output looks like this:
decrypted = "some-secres"
encrypted = "UARxWKJtEPzrYjQnGL4o8BxfJWI8FIoHCiEWnzk06et4yWwN1vovKRwhTT4XBMp7KkFnORQp4rTn_8V8wouWjdgpCIKaDcOTEDR7dBPnsg2Wj1NW-5rhLIo9BsZsdZ2auTQbzx5_6zz8LR-TGLFmdBjkoIt19AAeKaoDNA8R3cP09bRfmjN45xCS1RbutRgJNOu2U49YESXohcKEIq3PVR91dxUeevxdzs3rgxbK-UJfu-n9HTKtbYFh7jG8NBtlFnpD9KxeIy8NOfTyQ8Nqk6xYPtrU4LAGeMvftC79lKw6ahP9tV8z75YZTKmhPHcL9Wg26Jy-CK39tufdE3gFmA"
while experimenting with creation of secrets from already encrypted secrets I saw that the end of the base64 somehow seems to be mangled sometimes. That would be in line with the observation of the mangled last character of the secret with the output.
here is an integration test highlighting the issue: https://github.com/hashicorp/terraform-provider-azurerm/compare/main...pellepelster:terraform-provider-azurerm:main
I am not sure how the encrypt API of KeyVault is intended to be used, but this script using the plain REST API produces the same results:
#!/usr/bin/env bash
set -euo pipefail
ARM_SUBSCRIPTION_ID="xxx"
ARM_CLIENT_ID="xxx"
ARM_CLIENT_SECRET="xxx"
ARM_TENANT_ID="xxx"
VAULT_BASE_URL="https://xxx.vault.azure.net"
VAULT_KEY_NAME="xxx"
SECRET="some-secret"
ACCES_TOKEN=$(curl --silent -X POST -d "grant_type=client_credentials&client_id=${ARM_CLIENT_ID}&client_secret=${ARM_CLIENT_SECRET}&resource=https%3A%2F%2Fvault.azure.net" https://login.microsoftonline.com/${ARM_TENANT_ID}/oauth2/token | jq -r ".access_token")
ENCRYPT_RESPONSE=$(curl --silent -X POST -d "{ \"alg\": \"RSA1_5\", \"value\": \"${SECRET}\" }" -H "Authorization: Bearer ${ACCES_TOKEN}" -H "Content-Type: application/json" "${VAULT_BASE_URL}/keys/${VAULT_KEY_NAME}/encrypt?api-version=7.4")
ENCRYPTED=$(echo "${ENCRYPT_RESPONSE}" | jq -r ".value")
DECRYPT_RESPONSE=$(curl --silent -X POST -d "{ \"alg\": \"RSA1_5\", \"value\": \"${ENCRYPTED}\" }" -H "Authorization: Bearer ${ACCES_TOKEN}" -H "Content-Type: application/json" "${VAULT_BASE_URL}/keys/${VAULT_KEY_NAME}/decrypt?api-version=7.4" | jq)
DECRYPTED=$(echo "${DECRYPT_RESPONSE}" | jq -r ".value")
echo "secret was: '${SECRET}'"
echo "after encryption/decryption: '${DECRYPTED}'"
in my case:
secret was: 'some-secret'
after encryption/decryption: 'some-secres'
thanks @pellepelster for your informatin, i reproduced it and have the same result as yours. i have filed an issue in Azure-rest-api-spec project to see if the service team has any suggestions.
in my case:
secret was: 'some-secret' after encryption/decryption: 'some-secres'
Hi @pellepelster ,
Key Vault uses Base64 URL encoding for the content to be encrypted / decrypted.
Because each character in the base64 encoding represents 64 possible cases, and 2^6 = 64, so each base64 character encodes 6 bits of data, which is not always aligned with the 8-bit byte we are working with more directly.
In your case, "some-secres" "some-secret" "some-secreu" and "some-secrev" all represent the same byte array.
PS C:\Users\dalzhao> [convert]::FromBase64String("some+secres=") | Format-Hex
Path:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 B2 89 9E FA C7 9C AD EB ²úÇë
PS C:\Users\dalzhao> [convert]::FromBase64String("some+secret=") | Format-Hex
Path:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 B2 89 9E FA C7 9C AD EB ²úÇë
PS C:\Users\dalzhao> [convert]::FromBase64String("some+secreu=") | Format-Hex
Path:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 B2 89 9E FA C7 9C AD EB ²úÇë
PS C:\Users\dalzhao> [convert]::FromBase64String("some+secrev=") | Format-Hex
Path:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 B2 89 9E FA C7 9C AD EB ²úÇë
The 11 characters in "some-secret" contains 66 bits of information, but we only encode 64 bits or 8-byte of information, the extra 2 bit of information is redundant, allowing us to use any of the "stuv" as the last character.
If you want to encrypt "some-secret", you need to do the following.
Convert "some-secret" from string to byte array.
PS C:\Users\dalzhao> $bytes = [System.Text.Encoding]::UTF8.GetBytes("some-secret")
PS C:\Users\dalzhao> $bytes | Format-Hex
Path:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 73 6F 6D 65 2D 73 65 63 72 65 74 some-secret
2. Convert the byte-array into base64 url string
PS C:\Users\dalzhao> [convert]::ToBase64String($bytes) -replace [regex]::Escape('+'),'-' -replace '/','_' -replace '=' c29tZS1zZWNyZXQ
3. send it through the encrypt API.
@EZhao-Quest , Azure Key Vault uses base64-URL encoding in the response.
To convert base64 URL encoding back to base64 encoding, we need to replace the '-' with '+', '_', with '/', and then add either 1 or 2 padding characters to make the length a multiple of 4.
@EZhao-Quest , Azure Key Vault uses base64-URL encoding in the response.
To convert base64 URL encoding back to base64 encoding, we need to replace the '-' with '+', '_', with '/', and then add either 1 or 2 padding characters to make the length a multiple of 4.
@dalzhao does this mean we have to add the padding characters in client side?
another point is that passing a string that is not base64url encoded to encrypt API is also acceptable and then the result of decrypt API is not base64url encoded too? just like use some-secret
as the value of encrypt.
@dalzhao
Thanks for the detailed explanation. That is completely understandable from a technical point of view (and also in line with other solutions like AWS KMS or Hashicorp Vault). I think it would be beneficial to be a little more explicit about this in the docs (https://learn.microsoft.com/en-us/rest/api/keyvault/keys/encrypt/encrypt?tabs=HTTP) because value
is just described as string there.
Even more confusing in the sourcecode it is marked as base64 encoded string
@wuxu92, you have 2 points:
@dalzhao does this mean we have to add the padding characters in client side?
Yes. We don't expect customers to call the REST API directly. We expect the applications calling Azure Key Vault would take advantage of the Azure Key Vault SDK library, which should handle all the conversions.
another point is that passing a string that is not base64url encoded to encrypt API is also acceptable and then the result of decrypt API is not base64url encoded too? just like use
some-secret
as the value of encrypt.
I don't quite understand your comment. "some-secret" is a valid base-64 url encoded string. The 64 characters you can use in a base64 url encoded string are A-Z, a-z, 0-9, - (hyphen) and _ (underscore).
I don't quite understand your comment. "some-secret" is a valid base-64 url encoded string. The 64 characters you can use in a base64 url encoded string are A-Z, a-z, 0-9, - (hyphen) and _ (underscore).
thanks for your explanation, i understand now for some-secret
is a valid base64url encoded string.
We expect the applications calling Azure Key Vault would take advantage of the Azure Key Vault SDK library, which should handle all the conversions.
terraform use the go-azure-sdk to call the API directly so we have to handle encoding/decoding manually.
thanks again for your help!
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.
Is there an existing issue for this?
Community Note
d
Description
azurerm_key_vault_encrypted_value should return plain_text_value when I pass a encrypted_data .
user case:
we want to use terraform to auto create a azure keyvault secret to store senstive data.
we cannot put the plain text in our source code, we need put an encrpted string to the source code in github.
the expected behaviour is we put the encrpted data to data.azurerm_key_vault_encrypted_value, then get the plain text from data.azurerm_key_vault_encrypted_value.plain_text_value, pass to the value of azurerm_key_vault_secret
current azurerm_key_vault_encrypted_value only return an id. cannot get plain_text_value.
New or Affected Resource(s)/Data Source(s)
azurerm_key_vault_encrypted_value
Potential Terraform Configuration
References
No response