DynamoDB table resource creation failing due to incorrect configuration of key_schema argument

naveen-amzn commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Terraform v1.2.9 on darwin_arm64 provider registry.terraform.io/hashicorp/awscc v0.52.0

Affected Resource(s)

awscc_dynamodb_table

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "awscc_dynamodb_table" "main" {
  table_name = "dynamodb_table_name"
  key_schema = {
    attribute_name = "UserId"
    key_type       = "HASH"
  }

  attribute_definitions = [{
    attribute_name = "UserId"
    attribute_type = "S"
    },
    {
      attribute_name = "GameTitle"
      attribute_type = "S"
    },
    {
      attribute_name = "TopScore"
      attribute_type = "S"
  }]

  billing_mode = "PROVISIONED"
  provisioned_throughput = {
    read_capacity_units  = 20
    write_capacity_units = 20
  }
  deletion_protection_enabled = true
  sse_specification = {
    sse_enabled = true
    sse_type    = "AWSKMS"
  }

  time_to_live_specification = {
    attribute_name = "TimeToExist"
    enabled        = false
  }
  table_class = "STANDARD"
  tags = [{
    key   = "NAME"
    value = "dynamodb-table-1"
    },
    {
      key   = "Environment"
      value = "development"
  }]
}

Debug Output

Panic Output

Expected Behavior

Amazon Dynamodb table should have been created.

Actual Behavior

After creating Terraform plan with above configuration it gave following output.

awscc_dynamodb_table.main: Refreshing state... [id=dynamodb_table_name]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # awscc_dynamodb_table.main will be created
  + resource "awscc_dynamodb_table" "main" {
      + arn                                  = (known after apply)
      + attribute_definitions                = [
          + {
              + attribute_name = "UserId"
              + attribute_type = "S"
            },
          + {
              + attribute_name = "GameTitle"
              + attribute_type = "S"
            },
          + {
              + attribute_name = "TopScore"
              + attribute_type = "S"
            },
        ]
      + billing_mode                         = "PROVISIONED"
      + contributor_insights_specification   = {
          + enabled = (known after apply)
        }
      + deletion_protection_enabled          = true
      + global_secondary_indexes             = [
        ] -> (known after apply)
      + id                                   = (known after apply)
      + import_source_specification          = {
          + input_compression_type = (known after apply)
          + input_format           = (known after apply)
          + input_format_options   = {
              + csv = {
                  + delimiter   = (known after apply)
                  + header_list = (known after apply)
                }
            }
          + s3_bucket_source       = {
              + s3_bucket       = (known after apply)
              + s3_bucket_owner = (known after apply)
              + s3_key_prefix   = (known after apply)
            }
        }
      + key_schema                           = {
          + "attribute_name" = "UserId"
          + "key_type"       = "HASH"
        }
      + kinesis_stream_specification         = {
          + stream_arn = (known after apply)
        }
      + local_secondary_indexes              = [
        ] -> (known after apply)
      + point_in_time_recovery_specification = {
          + point_in_time_recovery_enabled = (known after apply)
        }
      + provisioned_throughput               = {
          + read_capacity_units  = 20
          + write_capacity_units = 20
        }
      + sse_specification                    = {
          + kms_master_key_id = (known after apply)
          + sse_enabled       = true
          + sse_type          = "AWSKMS"
        }
      + stream_arn                           = (known after apply)
      + stream_specification                 = {
          + stream_view_type = (known after apply)
        }
      + table_class                          = "STANDARD"
      + table_name                           = "dynamodb_table_name"
      + tags                                 = [
          + {
              + key   = "NAME"
              + value = "dynamodb-table-1"
            },
          + {
              + key   = "Environment"
              + value = "production"
            },
        ]
      + time_to_live_specification           = {
          + attribute_name = "TimeToExist"
          + enabled        = false
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

But when applying the changes, it failed with the below error

│ Error: AWS SDK Go Service Operation Incomplete
│ 
│   with awscc_dynamodb_table.main,
│   on dynamodb_table.tf line 1, in resource "awscc_dynamodb_table" "main":
│    1: resource "awscc_dynamodb_table" "main" {
│ 
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Invalid request provided: Unrecognized field
│ key_type in KeySchema. ErrorCode: InvalidRequest

Steps to Reproduce

terraform apply

│ Error: AWS SDK Go Service Operation Incomplete
│ 
│   with awscc_dynamodb_table.main,
│   on dynamodb_table.tf line 1, in resource "awscc_dynamodb_table" "main":
│    1: resource "awscc_dynamodb_table" "main" {
│ 
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Invalid request provided: Unrecognized field
│ key_typ

When I provided key_schema as list of map then awscc provided below error message

│   on main.tf line 3, in resource "awscc_dynamodb_table" "main":
│    3:   key_schema = [
│    4:     {
│    5:       attribute_name = "UserId"
│    6:       key_type       = "HASH"
│    7:     }
│    8:   ]
│ 
│ Inappropriate value for attribute "key_schema": map of string required.

Important Factoids

References

AWSCC schema - https://github.com/hashicorp/terraform-provider-awscc/blob/main/internal/aws/dynamodb/table_resource_gen.go#L472-L490 Cloudformation schema - https://github.com/hashicorp/terraform-provider-awscc/blob/ce1ad5105c8f463339880e0[…]internal/service/cloudformation/schemas/AWS_DynamoDB_Table.jso

0000

wellsiau-aws commented 1 year ago

Key Schema object: https://github.com/hashicorp/terraform-provider-awscc/blob/ce1ad5105c8f463339880e0dd2d00395cd2f40a6/internal/service/cloudformation/schemas/AWS_DynamoDB_Table.json#L135-L149

wellsiau-aws commented 1 year ago

AWSCC handler for DynamoDB Table: https://github.com/hashicorp/terraform-provider-awscc/blob/ce1ad5105c8f463339880e0dd2d00395cd2f40a6/internal/aws/dynamodb/table_resource_gen.go#L472C1-L474C22

Notice how the element type is string

        "key_schema": schema.MapAttribute{ /*START ATTRIBUTE*/
            ElementType: types.StringType,
            Required:    true,

wellsiau-aws commented 1 year ago

Perhaps this part of the Key Schema is causing confusion on the auto-generation because it declare either an array or an object.

https://github.com/hashicorp/terraform-provider-awscc/blob/ce1ad5105c8f463339880e0dd2d00395cd2f40a6/internal/service/cloudformation/schemas/AWS_DynamoDB_Table.json#L32-L45

novekm commented 1 year ago

Having this error too in latest version of provider (0.55.0)

kadrach commented 1 year ago

resource "awscc_dynamodb_table" "main" {
  table_name = "dynamodb_table_name"
  key_schema = {
    attribute_name = "UserId"
    key_type       = "HASH"
  }

  attribute_definitions = [{
    attribute_name = "UserId"
    attribute_type = "S"
  }]

  billing_mode = "PAY_PER_REQUEST"
}

rg "CloudControl DesiredState" output.log | jq '.value | fromjson'                                            
{
  "AttributeDefinitions": [
    {
      "AttributeName": "UserId",
      "AttributeType": "S"
    }
  ],
  "BillingMode": "PAY_PER_REQUEST",
  "KeySchema": {
    "attribute_name": "UserId",          # ⚠️ 
    "key_type": "HASH"                   # ⚠️ 
  },
  "TableName": "dynamodb_table_name"
}

With key_schema being treated as a map(string) the translator does not translate the Terraform names back to CloudFormation names.

quixoticmonk commented 5 months ago

@wellsiau-aws The current working version of the schema is below. The original example has an incorrect sse_type of AWSKMS.

The current schema enforces the members of key_schema to match the attribute_definitions which is from the expectation on the AWS::DynamoDB::Table spec. This shouldn't be the case.
key_schema is expecting a JSON string format and doesn't take the snake cased arguments.

resource "awscc_dynamodb_table" "main" {
  table_name = "dynamodb_table_name"
  key_schema = jsonencode([{
    AttributeName = "UserId"
    KeyType       = "HASH"
  }])

  attribute_definitions = [{
    attribute_name = "UserId"
    attribute_type = "S"
    }]

  billing_mode = "PROVISIONED"
  provisioned_throughput = {
    read_capacity_units  = 20
    write_capacity_units = 20
  }
  deletion_protection_enabled = true
  sse_specification = {
    sse_enabled = true
    sse_type    = "KMS"
  }

  time_to_live_specification = {
    attribute_name = "TimeToExist"
    enabled        = false
  }
  table_class = "STANDARD"

  tags = [{
    key   = "Modified By"
    value = "AWSCC"
  }]
}

wellsiau-aws commented 4 months ago

@kadrach @quixoticmonk thank you for your input!

I believe there are two separate problem here:

key_schema attribute being treated as map(string) as described here
CCAPI write handler requires that key_schema attribute value to matches the attribute_definitions as described here

@quixoticmonk , I suggest to open a new issue to record your problem since that also another upstream AWS issue.

hashicorp / terraform-provider-awscc