AWS - Kms auto unseal not working

[SankarGopal77]

Hi,

We are using HashiCorp vault helm chart 0.8.0 and vault image 1.5.4 version. We've deployed this in EKS version 1.18 . We managed to unseal vault by manually however when tried the following AWS Kms for auto unseal , It didn't work .

seal "awskms" { region = "eu-west-2" kms_key_id = "***" }

First we tried deploying the helm chart with above config and it deployed with seal Type awsKms and seal set to true. We then followed the steps mentioned in this link to unseal.

https://learn.hashicorp.com/tutorials/vault/autounseal-aws-kms

After unsealing, the seal type changed to "Shamir" and seal status set to false.

but when we restarted the vault pods, it gone to CrashLoopBackOff status with below error.

core: entering seal migration mode; Vault will not automatically unseal even if using an autoseal: from_barrier_type=shamir to_barrier_type=awskms`

Is AWS auto unsealing process available in open source version or do we need to use enterprise version?

[jasonodonnell]

Hi @SankarGopal77, can you try this example in our documentation: https://www.vaultproject.io/docs/platform/k8s/helm/run#amazon-eks-auto-unseal.

Additionally helm delete doesn't delete PVCs, so make sure you delete the PVCs if you're trying a fresh installation.

[SankarGopal77]

Hi @jasonodonnell , Many thanks for quick reply. Yes we followed exactly the same steps as per the link (https://learn.hashicorp.com/tutorials/vault/autounseal-aws-kms) , only difference is we use dynamoDB as our backend storage. This is our config. We use IAM roles to connect to EKS hence I believe we don't have set AWS access and secret key..

config: | ui = true

  listener "tcp" {
    tls_cert_file = "/vault/userconfig/vault-server-tls/vault.crt"
    tls_key_file  = "/vault/userconfig/vault-server-tls/vault.key"
    tls_client_ca_file = "/vault/userconfig/vault-server-tls/vault.ca"
    address = "[::]:8200"
    cluster_address = "[::]:8201"
  }

  storage "dynamodb" {
    ha_enabled = "true"
    region     = "eu-west-2"
    table      = "vault-dynamodb-devops"
  }

  seal "awskms" {
    region     = "eu-west-2"
    kms_key_id = "**************"
  }

As per my earlier comment, first time we are able to manually unseal Vault but then when we restart the pod it is throwing below error.

core: entering seal migration mode; Vault will not automatically unseal even if using an autoseal: from_barrier_type=shamir to_barrier_type=awskms`

Please correct me if I missed any other steps.

[tvoran]

One thing I'll mention is that you don't need to manually unseal before letting awskms handle the auto-unsealing. If operator init is run with the awskms seal config in place, vault should come up unsealed with awskms.

But it is also possible to migrate from shamir to auto unseal. That process is described in detail here: https://www.vaultproject.io/docs/concepts/seal#seal-migration

[pearsonhenri]

Ran into this as well-- @jasonodonnell 's note about PVCs not being deleted is what was tripping me up too! (Had previously installed and uninstalled a deployment without autoseal configured, and I suppose that state was persisted to the PVC.)

Deleting the PVC and truly starting from scratch set me right :)

[talonx]

@jasonodonnell Thank you for the note about PVCs - that's what I ran into as well and your comment helped.

[guidoffm]

Make sure to flush the DynamoDB table first!