[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"

[binacloud]

I'm deploying using Terraform, the latest Docker image Hashicorp Vault 1.11.0 on Amazon ECS, using DynamoDB as the backend.

Environment variables declared in container_definitions : VAULT_ADDR=https://0.0.0.0:8201/ VAULT_API_ADDR=https://127.0.0.1:8201 VAULT_SEAL_TYPE=awskms VAULT_AWSKMS_SEAL_KEY_ID=arn:aws:kms:<REGION>:<ACCOUNT_ID>:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee VAULT_UI=true VAULT_LOCAL_CONFIG={storage: {dynamodb: {ha_enabled: true, region: ${var.region}, table: ${aws_dynamodb_table.vault_dynamo.name} }}, listener: {tcp: { address: 0.0.0.0:8201, tls_disable: false, tls_cert_file: /mnt/efs/tls.crt, tls_key_file: /mnt/efs/tls.key } } }

Certificate and private key are mounted as EFS volume.

but at every deployment I'm getting the error below :

Storage: dynamodb (HA available)
Recovery Mode: false
Mlock: supported: true, enabled: true
Log Level: trace
Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "enabled")
Go Version: go1.17.11
Cluster Address: https://127.0.0.1:8201
Cgo: disabled
Api Address: https://127.0.0.1:8200
==> Vault server configuration:
==> Vault server started! Log data will stream in below:
Version Sha: ea296ccf58507b25051bc0597379c467046eb2f1
Version: Vault v1.11.0, built 2022-06-17T15:48:44Z

[INFO] core: stored unseal keys supported, attempting fetch
[DEBUG] would have sent systemd notification (systemd not present): notification=READY=1
[DEBUG] cluster listener addresses synthesized: cluster_addresses=[0.0.0.0:8201]
[INFO] core: Initializing version history cache for core
[DEBUG] storage.cache: creating LRU cache: size=0
[DEBUG] core: set config: sanitized config="{\"api_addr\":\"\",\"cache_size\":0,\"cluster_addr\":\"\",\"cluster_cipher_suites\":\"\",\"cluster_name\":\"\",\"default_lease_ttl\":0,\"default_max_request_duration\":0,\"disable_cache\":false,\"disable_clustering\":false,\"disable_indexing\":false,\"disable_mlock\":false,\"disable_performance_standby\":false,\"disable_printable_check\":false,\"disable_sealwrap\":false,\"disable_sentinel_trace\":false,\"enable_response_header_hostname\":false,\"enable_response_header_raft_node_id\":false,\"enable_ui\":false,\"listeners\":[{\"config\":{\"address\":\"0.0.0.0:8200\",\"tls_cert_file\":\"/mnt/efs/tls.crt\",\"tls_disable\":\"false\",\"tls_key_file\":\"/mnt/efs/tls.key\"},\"type\":\"tcp\"}],\"log_format\":\"unspecified\",\"log_level\":\"\",\"log_requests_level\":\"\",\"max_lease_ttl\":0,\"pid_file\":\"\",\"plugin_directory\":\"\",\"plugin_file_permissions\":0,\"plugin_file_uid\":0,\"raw_storage_endpoint\":false,\"seals\":[{\"disabled\":false,\"type\":\"awskms\"}],\"storage\":{\"cluster_addr\":\"\",\"disable_clustering\":false,\"redirect_addr\":\"\",\"type\":\"dynamodb\"}}"
[INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"
[INFO] core: stored unseal keys supported, attempting fetch
[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"
[INFO] core: stored unseal keys supported, attempting fetch
[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"
[INFO] core: stored unseal keys supported, attempting fetch
[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"
[INFO] core: stored unseal keys supported, attempting fetch
[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"
[INFO] core: stored unseal keys supported, attempting fetch
[WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"

The KMS key definitely exist and correct. When I try a false KMS key ID, it logs "key not found". When I try a KMS key ID from another AWS account, it logs "Access Denied"

The IAM policy of the KMS key is as permissive as possible, and also the ECS assumed task role is super permissive. Even if it's not I should see some permission or access related errors right?? So why is it giving this vague warning that's clearly an error ??

I've enabled "trace" level log and still nothing useful. This should be marked as [ERROR] because it fails the container after some time.

What does this error mean ??