[Bug]: TF confused of state of SSM parameter

[FransUrbo]

Terraform Core Version

v1.3.3

AWS Provider Version

4.35.0

Affected Resource(s)

• aws_ssm_parameter

Expected Behavior

Nothing. Parameter haven't change, nor is it about to..

Actual Behavior

TF fails and stops processing, both plan and apply are affected. Refuses to go past this.

Relevant Error/Panic Output Snippet

Error: error reading SSM Parameter (<prefix>-ecache-dbpass): InvalidKeyId: arn:aws:kms:<region>:<account>:key/<key_id> is pending deletion. (Service: AWSKMS; Status Code: 400; Error Code: KMSInvalidStateException; Request ID: e81be56c-fcd4-49e0-b882-40def8749a9a; Proxy: null)

Terraform Configuration Files

resource "random_password" "userpass" {
    length                              = 128
    special                             = false
    min_lower                           = 4
    min_upper                           = 4
    min_numeric                         = 4
}

# ===

resource "aws_ssm_parameter" "db_pass" {
    name                                = "${var.prefix}-ecache-dbpass"
    description                         = "Redis auth token/password"
    type                                = "SecureString"
    value                               = random_password.userpass.result
    key_id                              = aws_kms_key.redis.id

    depends_on = [random_password.userpass]
}

Steps to Reproduce

No idea, this "worked yesterday" and nothing have changed..

Debug Output

Looking at the state:

$ terraform state show module.redis.aws_ssm_parameter.db_pass
# module.redis.aws_ssm_parameter.db_pass:
resource "aws_ssm_parameter" "db_pass" {
    allowed_pattern = ""
    arn             = "arn:aws:ssm:<region>:<account>:parameter/<prefix>-ecache-dbpass"
    data_type       = "text"
    description     = "Redis auth token/password"
    id              = "<prefix>-ecache-dbpass"
    key_id          = "<key_id>"
    name            = "<prefix>-ecache-dbpass"
    tags            = {}
    tags_all        = {
        "environment" = "<prefix>"
        "location"    = "<prefix>"
    }
    tier            = "Standard"
    type            = "SecureString"
    value           = (sensitive value)
    version         = 1
}
$

Looking at the param with the aws command:

$ aws ssm get-parameter --name <prefix>-ecache-dbpass
{
    "Parameter": {
        "Name": "<prefix>-ecache-dbpass",
        "Type": "SecureString",
        "Value": "<big_long_value_string>",
        "Version": 1,
        "LastModifiedDate": "2022-09-29T08:17:33.253000+01:00",
        "ARN": "arn:aws:ssm:<region>:<account>:parameter/<prefix>-ecache-dbpass",
        "DataType": "text"
    }
}
$

Another way:

$ aws ssm describe-parameters | jq -r '.Parameters[] | select(.Name == "<prefix>-ecache-dbpass")'
{
  "Name": "<prefix>-ecache-dbpass",
  "Type": "SecureString",
  "KeyId": "<key_id>",
  "LastModifiedDate": "2022-09-29T08:17:33.253000+01:00",
  "LastModifiedUser": "arn:aws:sts::<account>:assumed-role/<my_federated_SSO_role>",
  "Description": "Redis auth token/password",
  "Version": 1,
  "Tier": "Standard",
  "Policies": [],
  "DataType": "text"
}
$

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

No response

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[Posnakidesd]

I'm having exact same issue. Somehow the KMS key was marked for deletion but referenced somewhere!