awscc_sagemaker_domain stuck in modifying state on consecutive applies

philipgebus commented 1 week ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Terraform v1.8.4

provider registry.terraform.io/hashicorp/awscc v1.3.0

Affected Resource(s)

awscc_sagemaker_domain

Terraform Configuration Files

resource "awscc_sagemaker_domain" "this" {
  domain_name = "awscc-test"
  auth_mode   = "IAM"

  vpc_id     = var.vpc_id
  subnet_ids = var.subnet_ids_private

  app_network_access_type = "PublicInternetOnly"

  default_user_settings = {
    execution_role         = awscc_iam_role.domain_exec.arn
    studio_web_portal   = "ENABLED"
    default_landing_uri = "studio::"
  }
}

Output

First run completes successfully:

# awscc_sagemaker_domain.this will be created
  + resource "awscc_sagemaker_domain" "this" {
      + app_network_access_type                        = "PublicInternetOnly"
      + app_security_group_management                  = (known after apply)
      + auth_mode                                      = "IAM"
      + default_space_settings                         = (known after apply)
      + default_user_settings                          = {
          + code_editor_app_settings         = (known after apply)
          + custom_file_system_configs       = (known after apply)
          + custom_posix_user_config         = (known after apply)
          + default_landing_uri              = "studio::"
          + execution_role                   = (known after apply)
          + jupyter_lab_app_settings         = (known after apply)
          + jupyter_server_app_settings      = (known after apply)
          + kernel_gateway_app_settings      = (known after apply)
          + r_session_app_settings           = (known after apply)
          + r_studio_server_pro_app_settings = (known after apply)
          + security_groups                  = (known after apply)
          + sharing_settings                 = (known after apply)
          + space_storage_settings           = (known after apply)
          + studio_web_portal                = "ENABLED"
        }
      + domain_arn                                     = (known after apply)
      + domain_id                                      = (known after apply)
      + domain_name                                    = "awscc-test"
      + domain_settings                                = (known after apply)
      + home_efs_file_system_id                        = (known after apply)
      + id                                             = (known after apply)
      + kms_key_id                                     = (known after apply)
      + security_group_id_for_domain_boundary          = (known after apply)
      + single_sign_on_application_arn                 = (known after apply)
      + single_sign_on_managed_application_instance_id = (known after apply)
      + subnet_ids                                     = [
          + "subnet-***",
          + "subnet-***",
        ]
      + tags                                           = (known after apply)
      + url                                            = (known after apply)
      + vpc_id                                         = "vpc-***"
    }
...
awscc_sagemaker_domain.this: Still creating... [1m30s elapsed]
awscc_sagemaker_domain.this: Still creating... [1m40s elapsed]
awscc_sagemaker_domain.this: Still creating... [1m50s elapsed]
awscc_sagemaker_domain.this: Still creating... [2m0s elapsed]
awscc_sagemaker_domain.this: Still creating... [2m10s elapsed]
awscc_sagemaker_domain.this: Still creating... [2m20s elapsed]
awscc_sagemaker_domain.this: Still creating... [2m30s elapsed]
awscc_sagemaker_domain.this: Still creating... [2m40s elapsed]
awscc_sagemaker_domain.this: Still creating... [2m50s elapsed]
awscc_sagemaker_domain.this: Still creating... [3m0s elapsed]
awscc_sagemaker_domain.this: Still creating... [3m10s elapsed]
awscc_sagemaker_domain.this: Still creating... [3m20s elapsed]
awscc_sagemaker_domain.this: Creation complete after 3m24s [id=d-fsn4utvvsndk]

Apply complete! Resources: 7 added, 0 changed, 0 destroyed.

Consecutive runs with the identical configuration trigger in-place updates for undefined attributes and get stuck:

# awscc_sagemaker_domain.this will be updated in-place
  ~ resource "awscc_sagemaker_domain" "this" {
      + app_security_group_management                  = (known after apply)
      + default_space_settings                         = (known after apply)
      ~ default_user_settings                          = {
          + code_editor_app_settings         = (known after apply)
          + custom_file_system_configs       = (known after apply)
          + custom_posix_user_config         = (known after apply)
          + jupyter_lab_app_settings         = (known after apply)
          + jupyter_server_app_settings      = (known after apply)
          + kernel_gateway_app_settings      = (known after apply)
          + r_session_app_settings           = (known after apply)
          + r_studio_server_pro_app_settings = (known after apply)
          + security_groups                  = (known after apply)
          + sharing_settings                 = (known after apply)
            # (4 unchanged attributes hidden)
        }
      + domain_settings                                = (known after apply)
        id                                             = "d-fsn4utvvsndk"
      + kms_key_id                                     = (known after apply)
      + security_group_id_for_domain_boundary          = (known after apply)
      + single_sign_on_application_arn                 = (known after apply)
      + single_sign_on_managed_application_instance_id = (known after apply)
      + tags                                           = (known after apply)
        # (9 unchanged attributes hidden)
    }
...
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 10m20s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 10m30s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 10m40s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 10m50s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 11m0s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 11m10s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 11m20s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 11m30s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 11m40s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 11m50s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 12m0s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 12m10s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 12m20s elapsed]
awscc_sagemaker_domain.this: Still modifying... [id=d-fsn4utvvsndk, 12m30s elapsed]

Expected Behavior

terraform apply solely updates the _awscc_sagemakerdomain resource if actual configuration changes have been made
terraform apply on consecutive applies does not get stuck in modifying state

Actual Behavior

On the first run, terraform apply executes successfully
On consecutive runs, terraform apply updates the _awscc_sagemakerdomain resource although no updates have been made and gets stuck in modifying state.

Steps to Reproduce

terraform apply
terraform apply on identical resource configuration

quixoticmonk commented 3 days ago

@wellsiau-aws

Prior state

{
  "app_network_access_type": "PublicInternetOnly",
  "app_security_group_management": null,
  "auth_mode": "IAM",
  "default_space_settings": null,
  "default_user_settings": {
    "code_editor_app_settings": null,
    "custom_file_system_configs": null,
    "custom_posix_user_config": null,
    "default_landing_uri": "studio::",
    "execution_role": "arn:aws:iam::###########:role/example",
    "jupyter_lab_app_settings": null,
    "jupyter_server_app_settings": null,
    "kernel_gateway_app_settings": null,
    "r_session_app_settings": null,
    "r_studio_server_pro_app_settings": null,
    "security_groups": null,
    "sharing_settings": null,
    "space_storage_settings": {
      "default_ebs_storage_settings": {
        "default_ebs_volume_size_in_gb": 5,
        "maximum_ebs_volume_size_in_gb": 100
      }
    },
    "studio_web_portal": "ENABLED"
  },
  "domain_arn": "arn:aws:sagemaker:us-east-1:###########:domain/d-gkp3bop4nqmv",
  "domain_id": "d-gkp3bop4nqmv",
  "domain_name": "awscc-test",
  "domain_settings": null,
  "home_efs_file_system_id": "fs-03a6e13630756c41c",
  "id": "d-gkp3bop4nqmv",
  "kms_key_id": null,
  "security_group_id_for_domain_boundary": null,
  "single_sign_on_application_arn": null,
  "single_sign_on_managed_application_instance_id": null,
  "subnet_ids": [
    "subnet-0e3cd1df31dea5e9c"
  ],
  "tags": null,
  "url": "https://d-gkp3bop4nqmv.studio.us-east-1.sagemaker.aws",
  "vpc_id": "vpc-091e289e155590a6f"
}

ProposedNewState

{
  "app_network_access_type": "PublicInternetOnly",
  "app_security_group_management": null,
  "auth_mode": "IAM",
  "default_space_settings": null,
  "default_user_settings": {
    "code_editor_app_settings": null,
    "custom_file_system_configs": null,
    "custom_posix_user_config": null,
    "default_landing_uri": "studio::",
    "execution_role": "arn:aws:iam::###########:role/example",
    "jupyter_lab_app_settings": null,
    "jupyter_server_app_settings": null,
    "kernel_gateway_app_settings": null,
    "r_session_app_settings": null,
    "r_studio_server_pro_app_settings": null,
    "security_groups": null,
    "sharing_settings": null,
    "space_storage_settings": null,
    "studio_web_portal": "ENABLED"
  },
  "domain_arn": "arn:aws:sagemaker:us-east-1:###########:domain/d-gkp3bop4nqmv",
  "domain_id": "d-gkp3bop4nqmv",
  "domain_name": "awscc-test",
  "domain_settings": null,
  "home_efs_file_system_id": "fs-03a6e13630756c41c",
  "id": "d-gkp3bop4nqmv",
  "kms_key_id": null,
  "security_group_id_for_domain_boundary": null,
  "single_sign_on_application_arn": null,
  "single_sign_on_managed_application_instance_id": null,
  "subnet_ids": [
    "subnet-0e3cd1df31dea5e9c"
  ],
  "tags": null,
  "url": "https://d-gkp3bop4nqmv.studio.us-east-1.sagemaker.aws",
  "vpc_id": "vpc-091e289e155590a6f"
}

{
  "app_network_access_type": "PublicInternetOnly",
  "app_security_group_management": "\u0000",
  "auth_mode": "IAM",
  "default_space_settings": "\u0000",
  "default_user_settings": {
    "code_editor_app_settings": "\u0000",
    "custom_file_system_configs": "\u0000",
    "custom_posix_user_config": "\u0000",
    "default_landing_uri": "studio::",
    "execution_role": "arn:aws:iam::###########:role/example",
    "jupyter_lab_app_settings": "\u0000",
    "jupyter_server_app_settings": "\u0000",
    "kernel_gateway_app_settings": "\u0000",
    "r_session_app_settings": "\u0000",
    "r_studio_server_pro_app_settings": "\u0000",
    "security_groups": "\u0000",
    "sharing_settings": "\u0000",
    "space_storage_settings": {
      "default_ebs_storage_settings": {
        "default_ebs_volume_size_in_gb": 5,
        "maximum_ebs_volume_size_in_gb": 100
      }
    },
    "studio_web_portal": "ENABLED"
  },
  "domain_arn": "arn:aws:sagemaker:us-east-1:###########:domain/d-gkp3bop4nqmv",
  "domain_id": "d-gkp3bop4nqmv",
  "domain_name": "awscc-test",
  "domain_settings": "\u0000",
  "home_efs_file_system_id": "fs-03a6e13630756c41c",
  "id": "d-gkp3bop4nqmv",
  "kms_key_id": "\u0000",
  "security_group_id_for_domain_boundary": "\u0000",
  "single_sign_on_application_arn": "\u0000",
  "single_sign_on_managed_application_instance_id": "\u0000",
  "subnet_ids": [
    "subnet-0e3cd1df31dea5e9c"
  ],
  "tags": "\u0000",
  "url": "https://d-gkp3bop4nqmv.studio.us-east-1.sagemaker.aws",
  "vpc_id": "vpc-091e289e155590a6f"
}

Replacing with the Required:true set on the nested block space_storage_settings


"space_storage_settings": schema.SingleNestedAttribute{ /*START ATTRIBUTE*/
                Attributes: map[string]schema.Attribute{ /*START SCHEMA*/
                    // Property: DefaultEbsStorageSettings
                    "default_ebs_storage_settings": schema.SingleNestedAttribute{ /*START ATTRIBUTE*/
                        Attributes: map[string]schema.Attribute{ /*START SCHEMA*/
                            // Property: DefaultEbsVolumeSizeInGb
                            "default_ebs_volume_size_in_gb": schema.Int64Attribute{ /*START ATTRIBUTE*/
                                Description: "Default size of the Amazon EBS volume in Gb",
                                // Required:    true,
                                Optional: true,
                                Computed: true,
                                Validators: []validator.Int64{ /*START VALIDATORS*/
                                    int64validator.Between(5, 16384),
                                }, /*END VALIDATORS*/
                            }, /*END ATTRIBUTE*/
                            // Property: MaximumEbsVolumeSizeInGb
                            "maximum_ebs_volume_size_in_gb": schema.Int64Attribute{ /*START ATTRIBUTE*/
                                Description: "Maximum size of the Amazon EBS volume in Gb. Must be greater than or equal to the DefaultEbsVolumeSizeInGb.",
                                // Required:    true,
                                Optional: true,
                                Computed: true,
                                Validators: []validator.Int64{ /*START VALIDATORS*/
                                    int64validator.Between(5, 16384),
                                }, /*END VALIDATORS*/
                            }, /*END ATTRIBUTE*/
                        }, /*END SCHEMA*/
                        Description: "Properties related to the Amazon Elastic Block Store volume. Must be provided if storage type is Amazon EBS and must not be provided if storage type is not Amazon EBS",
                        Optional:    true,
                        Computed:    true,
                        PlanModifiers: []planmodifier.Object{ /*START PLAN MODIFIERS*/
                            objectplanmodifier.UseStateForUnknown(),
                        }, /*END PLAN MODIFIERS*/
                    }, /*END ATTRIBUTE*/
                }, /*END SCHEMA*/
                Description: "Default storage settings for a space.",
                Optional:    true,
                Computed:    true,
                PlanModifiers: []planmodifier.Object{ /*START PLAN MODIFIERS*/
                    objectplanmodifier.UseStateForUnknown(),
                }, /*END PLAN MODIFIERS*/
            }, /*END ATTRIBUTE*/
            // Property: StudioWebPortal
            "studio_web_portal": schema.StringAttribute{ /*START ATTRIBUTE*/
                Description: "Indicates whether the Studio experience is available to users. If not, users cannot access Studio.",
                Optional:    true,
                Computed:    true,
                Validators: []validator.String{ /*START VALIDATORS*/
                    stringvalidator.OneOf(
                        "ENABLED",
                        "DISABLED",
                    ),
                }, /*END VALIDATORS*/
                PlanModifiers: []planmodifier.String{ /*START PLAN MODIFIERS*/
                    stringplanmodifier.UseStateForUnknown(),
                }, /*END PLAN MODIFIERS*/
            }, /*END ATTRIBUTE*/
        }, /*END SCHEMA*/
        Description: "The default user settings.",
        Required:    true,
    }, /*END ATTRIBUTE*/

data.aws_iam_policy_document.example: Reading... data.aws_iam_policy_document.example: Read complete after 0s [id=1147345228] aws_iam_role.example: Refreshing state... [id=example] awscc_sagemaker_domain.this: Refreshing state... [id=d-gkp3bop4nqmv]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

quixoticmonk commented 3 days ago

Thank you for opening this issue, @philipgebus . Tracking this as a bug similar to https://github.com/hashicorp/terraform-provider-awscc/issues/1216#issuecomment-2172453750

hashicorp / terraform-provider-awscc