awscc_s3_bucket: Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: . ErrorCode: InternalFailure

[rchildress87]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

$ terraform -v
Terraform v1.3.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.36.1
+ provider registry.terraform.io/hashicorp/awscc v0.35.0

Affected Resource(s)

• awscc_s3_bucket

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

terraform {
  required_version = ">= 0.13.1"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.9"
    }
    awscc = {
      source  = "hashicorp/awscc"
      version = "0.35.0"
    }
  }
}

resource "awscc_s3_bucket" "this" {
}

Debug Output

https://gist.github.com/rchildress87/d78aab13428edc0cbf60cf0465351e26

Expected Behavior

There are no required arguments documented for the awscc_s3_bucket resource. A new bucket should have been created with a randomly generated name as documented here and here.

Actual Behavior

$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # awscc_s3_bucket.this will be created
  + resource "awscc_s3_bucket" "this" {
      + accelerate_configuration           = {
          + acceleration_status = (known after apply)
        }
      + access_control                     = (known after apply)
      + analytics_configurations           = [
        ] -> (known after apply)
      + arn                                = (known after apply)
      + bucket_encryption                  = {
          + server_side_encryption_configuration = [
            ] -> (known after apply)
        }
      + bucket_name                        = (known after apply)
      + cors_configuration                 = {
          + cors_rules = [
            ] -> (known after apply)
        }
      + domain_name                        = (known after apply)
      + dual_stack_domain_name             = (known after apply)
      + id                                 = (known after apply)
      + intelligent_tiering_configurations = [
        ] -> (known after apply)
      + inventory_configurations           = [
        ] -> (known after apply)
      + lifecycle_configuration            = {
          + rules = [
            ] -> (known after apply)
        }
      + logging_configuration              = {
          + destination_bucket_name = (known after apply)
          + log_file_prefix         = (known after apply)
        }
      + metrics_configurations             = [
        ] -> (known after apply)
      + notification_configuration         = {
          + event_bridge_configuration = {
              + event_bridge_enabled = (known after apply)
            }
          + lambda_configurations      = [
            ] -> (known after apply)
          + queue_configurations       = [
            ] -> (known after apply)
          + topic_configurations       = [
            ] -> (known after apply)
        }
      + object_lock_configuration          = {
          + object_lock_enabled = (known after apply)
          + rule                = {
              + default_retention = {
                  + days  = (known after apply)
                  + mode  = (known after apply)
                  + years = (known after apply)
                }
            }
        }
      + object_lock_enabled                = (known after apply)
      + ownership_controls                 = {
          + rules = [
            ] -> (known after apply)
        }
      + public_access_block_configuration  = {
          + block_public_acls       = (known after apply)
          + block_public_policy     = (known after apply)
          + ignore_public_acls      = (known after apply)
          + restrict_public_buckets = (known after apply)
        }
      + regional_domain_name               = (known after apply)
      + replication_configuration          = {
          + role  = (known after apply)
          + rules = [
            ] -> (known after apply)
        }
      + versioning_configuration           = {
          + status = (known after apply)
        }
      + website_configuration              = {
          + error_document           = (known after apply)
          + index_document           = (known after apply)
          + redirect_all_requests_to = {
              + host_name = (known after apply)
              + protocol  = (known after apply)
            }
          + routing_rules            = [
            ] -> (known after apply)
        }
      + website_url                        = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

awscc_s3_bucket.this: Creating...
╷
│ Error: AWS SDK Go Service Operation Incomplete
│ 
│   with awscc_s3_bucket.this,
│   on main.tf line 16, in resource "awscc_s3_bucket" "this":
│   16: resource "awscc_s3_bucket" "this" {
│ 
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: . ErrorCode: InternalFailure

Steps to Reproduce

1. terraform apply

Important Factoids

• It is possible this is an issue with the actual Cloud Control API. I have not attempted to create the resource using the AWS CLI.

[rchildress87]

I now believe this issue is most likely caused by a bug in the AWS Cloud Control API.

$ aws cloudcontrol create-resource --type-name "AWS::S3::Bucket" --desired-state "{}"
{
    "ProgressEvent": {
        "TypeName": "AWS::S3::Bucket",
        "RequestToken": "4cad2849-512e-4f39-991e-90040fbba509",
        "Operation": "CREATE",
        "OperationStatus": "IN_PROGRESS",
        "EventTime": "2022-10-21T18:42:47.219000-04:00"
    }
}

$ aws cloudcontrol get-resource-request-status --request-token "4cad2849-512e-4f39-991e-90040fbba509"
{
    "ProgressEvent": {
        "TypeName": "AWS::S3::Bucket",
        "RequestToken": "4cad2849-512e-4f39-991e-90040fbba509",
        "Operation": "CREATE",
        "OperationStatus": "FAILED",
        "EventTime": "2022-10-21T18:42:47.684000-04:00",
        "ErrorCode": "InternalFailure"
    }
}

Can anyone else confirm if my test was sufficient and if they are able to reproduce?

[rchildress87]

After reaching out to AWS Support to confirm the issue, it now appears that the Terraform provider's documentation for awscc_s3_bucket is inaccurate. While I do not believe I have access to the official Cloud Control resource schemas to confirm, it appears that other Cloud Control API partners are privy to Cloud Control API's requirement that a bucket name must be provided. For an example, see the Pulumi documentation. This would lead me to believe that Pulumi is using a different source to retrieve a resource's schema or that they are somewhat manually preparing the documentation.

For more context, here is an excerpt from my conversation with AWS Support:

Although the AWS::S3::Bucket[2] resource type in CloudFormation does not require you to specify a BucketName property, the underlying CreateBucket S3 API call[3] does require a BucketName to be supplied. In CloudFormation, this is handled by generating a globally unique bucket name based on the Logical ID of the AWS::S3::Bucket resource that is defined in your CloudFormation template. However, when using Cloud Control API, you do not provide a Logical ID of the resource like you would in a CloudFormation template, so the way the CreateResource call is handled by Cloud Control API is not 100% equivalent to how the CloudFormation Resource Handler works.

[2] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html [3] https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html

I'm curious now if a) the Cloud Control API resource schemas are inaccurate or b) the AWSCC provider documentation is generated from a less-than-ideal source. Can anyone share any helpful information?

[rchildress87]

Here are some more details from AWS Support that I received today to better explain what is going on. The last two paragraphs seem most important:

Hello,

Thank you for your patience while I looked into your question.

Regarding the Schema information, this is publicly available, and you can access it using the describe-type command (see below for an example using the AWS::S3::Bucket Resource Type):

aws cloudformation describe-type --type RESOURCE --type-name AWS::S3::Bucket --query Schema --output text

You will see in this schema output the following information for the \"BucketName\" property:

\"BucketName\": {
\"description\": \"A name for the bucket. If you don't specify a name, AWS CloudFormation generates a unique > physical ID and uses that ID for the bucket name.\",

This is identical to what is referred to in the Terraform documentation. However, Cloud Control API adds a bit of extra behavior on top of the underlying CloudFormation Resource Schema.

Specifically, the Cloud Control API requires the user to specify the primary identifier when interacting with a resource[1]. You can find the primary identifier for a given resource in the schema retrieved from the describe-type command.

Specifically, the primary identifier for the AWS::S3::Bucket resource type is the BucketName property, as seen below:

\"primaryIdentifier\": [
\"/properties/BucketName\"
],

This means, while the underlying CloudFormation schema is technically correct (BucketName is not required for the CloudFormation resource handler), the fact that the BucketName is listed in the Schema as the primaryIdentifier means that the Cloud Control API will enforce its presence in the create-resource command before CloudFormation handler even comes into the picture, which is why you see the InternalFailure message when trying to use create-resource for an S3 bucket with no BucketName provided.

It seems that the Pulumi documentation has accounted for this behavior in their documentation, whereas the Terraform documentation seems to have pulled directly from the CloudFormation resource type schema without accounting for the extra enforcement behavior that Cloud Control API introduces, which would explain the discrepancy observed between their provider documentation pages.

References: [1] https://docs.aws.amazon.com/cloudcontrolapi/latest/userguide/resource-identifier.html

This response indicates to me that any primary identifier, no matter if it is marked required in the schema or not, should be marked required in the provider's documentation. Thoughts?

[breathingdust]

Hi @rchildress87! At present we do not manually annotate documentation but that is something we do plan to allow in some fashion in the future. This does seem like it is a bug on the AWS side to us, so we will reach out to the Cloud Control team to confirm.

[ewbankkit]

the Cloud Control API will enforce its presence in the create-resource command before CloudFormation handler even comes into the picture

must be incorrect in general because for many resources the primary identifier is not known until after the resource has been created, think for example of the EC2 Instance ID or VPC ID. This does seem to be a bug in the schema.

Opened https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1405.

[breathingdust]

AWS have deployed a fix for this and I have verified that creating a bucket with the following config now succeeds:

resource "awscc_s3_bucket" "name" {
}

terraform apply --auto-approve

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # awscc_s3_bucket.name will be created
  + resource "awscc_s3_bucket" "name" {
      + accelerate_configuration           = {
          + acceleration_status = (known after apply)
        } -> (known after apply)
      + access_control                     = (known after apply)
      + analytics_configurations           = [
        ] -> (known after apply)
      + arn                                = (known after apply)
      + bucket_encryption                  = {
          + server_side_encryption_configuration = [
            ] -> (known after apply)
        } -> (known after apply)
      + bucket_name                        = (known after apply)
      + cors_configuration                 = {
          + cors_rules = [
            ] -> (known after apply)
        } -> (known after apply)
      + domain_name                        = (known after apply)
      + dual_stack_domain_name             = (known after apply)
      + id                                 = (known after apply)
      + intelligent_tiering_configurations = [
        ] -> (known after apply)
      + inventory_configurations           = [
        ] -> (known after apply)
      + lifecycle_configuration            = {
          + rules = [
            ] -> (known after apply)
        } -> (known after apply)
      + logging_configuration              = {
          + destination_bucket_name = (known after apply)
          + log_file_prefix         = (known after apply)
        } -> (known after apply)
      + metrics_configurations             = [
        ] -> (known after apply)
      + notification_configuration         = {
          + event_bridge_configuration = {
              + event_bridge_enabled = (known after apply)
            } -> (known after apply)
          + lambda_configurations      = [
            ] -> (known after apply)
          + queue_configurations       = [
            ] -> (known after apply)
          + topic_configurations       = [
            ] -> (known after apply)
        } -> (known after apply)
      + object_lock_configuration          = {
          + object_lock_enabled = (known after apply)
          + rule                = {
              + default_retention = {
                  + days  = (known after apply)
                  + mode  = (known after apply)
                  + years = (known after apply)
                } -> (known after apply)
            } -> (known after apply)
        } -> (known after apply)
      + object_lock_enabled                = (known after apply)
      + ownership_controls                 = {
          + rules = [
            ] -> (known after apply)
        } -> (known after apply)
      + public_access_block_configuration  = {
          + block_public_acls       = (known after apply)
          + block_public_policy     = (known after apply)
          + ignore_public_acls      = (known after apply)
          + restrict_public_buckets = (known after apply)
        } -> (known after apply)
      + regional_domain_name               = (known after apply)
      + replication_configuration          = {
          + role  = (known after apply)
          + rules = [
            ] -> (known after apply)
        } -> (known after apply)
      + tags                               = [
        ] -> (known after apply)
      + versioning_configuration           = {
          + status = (known after apply)
        } -> (known after apply)
      + website_configuration              = {
          + error_document           = (known after apply)
          + index_document           = (known after apply)
          + redirect_all_requests_to = {
              + host_name = (known after apply)
              + protocol  = (known after apply)
            } -> (known after apply)
          + routing_rules            = [
            ] -> (known after apply)
        } -> (known after apply)
      + website_url                        = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
awscc_s3_bucket.name: Creating...
awscc_s3_bucket.name: Still creating... [10s elapsed]
awscc_s3_bucket.name: Still creating... [20s elapsed]
awscc_s3_bucket.name: Creation complete after 24s [id=9z50r49dqceqxzeaazqotwiey-qlskzqlrewrz]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.