Store sensitive attributes as HSM secret value [output attributes]

[LaurentLesle]

Customers have some regulated use cases and just marking "secret" attributes as sensitive is a security breach for the organisation as their policy prevent those security tokens, passwords or pass-phrases to be stored outside of a HSM (secret store).

Some API calls are returning sensitive information (passwords, authorisation keys, crypto objects...) that should not be persisted in the tfstate.

Design goals:

• Supporting a pluggable approach where over time additional HSM (secret store) can be added like Hashicorp vault secrets, kubernetes secrets...
• Initial HSM support with Azure Key vault, Azure managed HSM, Azure dedicated HSM

Implementation directions:

• Add an optional attribute hsm_output_attrs to define the attributes to store in an external HSM as secrets instead of keeping them in the output of the object.
• Some secrets may have an expiration date
• Add an optional read-only attribute hsm_attributes that will output a map of objects with the reference to the secret uri

Challenges:

• Some API returns the secret value in the response payload on successful creation. On sub-sequent refresh the attribute is not present (e.g password) and cannot be updated in the HSM. This is a valid use case which should not trigger a destroy/recreate of the resource
• Some API returns always an attribute that should not be stored in the tfstate but kept in the HSM. On refresh, if the value of this attribute has changed then it will be updated in the HSM.

This issue describes the desired state of that feature.

resource "restful_resource" "create_secure_object" {
  path = format("...")
  create_method = "PUT"
  body = jsonencode({
    name = "ccc"
    ipsecpolicy = {
    }
  })

 // encryption_key  and passphrase output attribute will be removed from the output attribute and therefore not stored in the tfstate
 hsm_output_attrs = {

  encryption_key = {
     provider = azure.keyvault
     definition = {
       endpoint = "resource id of the keyvault"
       // HSM security context can inherit from the provider context or provide a specific block in the provider to set the HSM security context.
     }
     // remove the encryption_key from the body (json path)
     path = "body.encryption_key"
     // and store that value in the HSM with secret name = "encryption-key" and secret value = "body.encryption_key" (value of the json path) 
     secret_name = "encryption-key"

     // Optional (json_path)
     expiration_date = "body.encryption_key_end_date"
   }

   passphrase = {
     provider = hashicorp.vault
     definition = {
       endpoint = "endpoint to connect to Hashicorp vault"
     }
     path = "body.ipsecpolicy.passphrase"
     secret_name = "passphrase"
   }
 }

}

After apply execution it is expected that the output attribute will have "body.encryption_key" and "body.ipsecpolicy.passphrase" removed from the json.

A new output (read-only) attribute is available with the resource path of the secret

 output = {
   // api response json with hsm_output_attrs removed
  }

 hsm_attributes = {
   encryption_key = {
    provider = azure.keyvault
    secret_uri = "resource id of the un-versioned secret"
    secret_uri_versioned = ""
   }

   passphrase = {
    provider = hashicorp.vault
    secret_uri = "path of the secret"
   }
 }

[magodo]

@LaurentLesle Thank you for submitting this interesting feature request.

My first implication is maybe we can simply storing the state file to a HSM that comply with the organization's policy. I'm not sure whether http backend would help.

One challenge for:

Some API returns always an attribute that should not be stored in the tfstate but kept in the HSM. On refresh, if the value of this attribute has changed then it will be updated in the HSM.

This implies that we only store the secret pointer in the state. But the provider can not conditionally wipe attributes on read, i.e. it either always wipe/no wipe an attribute, given how the SDK supports. What is asked here is that we wipe the secret attribute during the read for refresh, but keep it during the read for the follow-up diff against the config.

[LaurentLesle]

Hi @magodo this requirement is more to segregate the tfstate store from the secrets that must be stored into an external HSM. The goal is to prevent secrets that would trigger a "non-compliance" or "security breach" because the secret is stored in the tfstate json. Storing the tfstate in an HSM or encryption the HSM would not address the previous statement.

One way to address the challenge provided by the SDK would be to remove always and follow something like that to handle the HSM flow:

[magodo]

@LaurentLesle Do you mean we shall always just ignore the secrest when generating a diff?

[LaurentLesle]

Yes for the tfstate flow.

But always run the HSM flow and adjust the response depending on plan, apply or destroy