Support Importing _prefix Arguments

bflad commented 5 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When importing Terraform resources that were created by Terraform using our standard name prefixing arguments (e.g. name_prefix), those resources must currently be imported with the configuration set to one of the following to prevent unexpected differences:

Not to include name and name_prefix
name set to the full name including the unique ID suffix (e.g. foo-20180130221122509800000001)
ignore_changes configured for name_prefix

All of which incorrectly match the original configuration which created it.

If name_prefix is configured for the imported resource, Terraform will show a (sometimes destructive) difference on the first plan:

name_prefix: "" => "example" (forces new resource)

Unless the lifecycle configuration block ignore_changes argument is used for the resource:

resource "aws_XXX" "example" {
  # ... other configuration ...
  name_prefix = "example"

  lifecycle {
    ignore_changes = ["name_prefix"]
  }
}

This problem can be reproduced with something like the following configuration and sequence of actions:

resource "aws_secretsmanager_secret" "example" {
  name_prefix = "example"
}

$ terraform apply
$ terraform state rm aws_secretsmanager_secret.example
$ terraform import aws_secretsmanager_secret.example XXX

We can detect this situation during import by checking if the naming suffix matches the standard resource.UniqueId() pattern and doesn't match the default resource.UniqueIdPrefix, which would suggest that name_prefix was configured for the resource that created it. If so, call d.Set("name_prefix", /*...*/) with the suffix stripped during the import function.

Acceptance testing for this can be done by removing these prefix attributes from ImportStateVerifyIgnore.

Some example logic:

func hasResourceUniqueIdPrefix(s string) bool {
  return strings.HasPrefix(s, resource.UniqueIdPrefix)
}

func hasResourceUniqueIdSuffix(s string) bool {
  suffixRegexp := regexp.MustCompile(fmt.Sprintf("\\d{%d}$", resource.UniqueIDSuffixLength))
  return suffixRegexp.MatchString(s)
}

func resourceNamingPrefix(name string) *string {
  if !hasResourceUniqueIdSuffix(name) {
    return nil
  }

  if hasResourceUniqueIdPrefix(name) {
    return nil
  }

  namePrefixIndex := len(name)-resource.UniqueIDSuffixLength

  if namePrefixIndex <= 0 {
    return nil
  }

  namePrefix := name[:namePrefixIndex]

  return &namePrefix
}

// example implementation in a resource given the import ID is a name
Importer: &schema.ResourceImporter{
    State: func(d *schema.ResourceData, meta interface{}) ([]*schema.ResourceData, error) {
        d.Set("name_prefix", resourceNamingPrefix(d.Id()))

        return []*schema.ResourceData{d}, nil
    },
},

Affected Resource(s)

Any resource implementing a XXX_prefix argument, e.g.

[ ] aws_autoscaling_group (name_prefix)
[ ] aws_budgets_budget (name_prefix)
[ ] aws_cloudfront_public_key (name_prefix)
[ ] aws_cloudwatch_event_rule (name_prefix)
[ ] aws_cloudwatch_log_group (name_prefix)
[ ] aws_db_event_subscription (name_prefix)
[ ] aws_db_instance (identifier_prefix)
[ ] aws_db_option_group (name_prefix)
[ ] aws_db_parameter_group (name_prefix)
[ ] aws_db_subnet_group (name_prefix)
[ ] aws_docdb_cluster (cluster_identifier_prefix)
[ ] aws_docdb_cluster_instance (identifier_prefix)
[ ] aws_docdb_cluster_parameter_group (name_prefix)
[ ] aws_docdb_subnet_group (name_prefix)
[ ] aws_elb (name_prefix)
[ ] aws_emr_security_configuration (name_prefix)
[ ] aws_iam_group_policy (name_prefix)
[ ] aws_iam_instance_profile (name_prefix)
[ ] aws_iam_policy (name_prefix)
[ ] aws_iam_role (name_prefix)
[ ] aws_iam_role_policy (name_prefix)
[ ] aws_iam_server_certificate (name_prefix)
[ ] aws_iam_user_policy (name_prefix)
[ ] aws_key_pair (key_name_prefix)
[ ] aws_kms_alias (name_prefix)
[ ] aws_lambda_permission (statement_id_prefix)
[ ] aws_launch_configuration (name_prefix)
[ ] aws_launch_template (name_prefix)
[ ] aws_lb (name_prefix)
[ ] aws_lb_target_group (name_prefix)
[ ] aws_lightsail_key_pair (name_prefix)
[ ] aws_neptune_cluster (cluster_identifier_prefix)
[ ] aws_neptune_cluster_instance (identifier_prefix)
[ ] aws_neptune_cluster_parameter_group (name_prefix)
[ ] aws_neptune_event_subscription (name_prefix)
[ ] aws_neptune_subnet_group (name_prefix)
[ ] aws_pinpoint_app (name_prefix)
[ ] aws_rds_cluster (cluster_identifier_prefix)
[ ] aws_rds_cluster_instance (identifier_prefix)
[ ] aws_rds_cluster_parameter_group (name_prefix)
[ ] aws_s3_bucket (bucket_prefix)
[ ] aws_secretsmanager_secret (name_prefix)
[x] aws_security_group (name_prefix)
[ ] aws_sns_topic (name_prefix)
[ ] aws_sqs_queue (name_prefix)
[ ] aws_swf_domain (name_prefix)

References

3206

bflad commented 4 years ago

An implementation of the proposal can be found now in the aws/internal/naming package with #12052 showing how it was done for the aws_security_group resource. The Contributing Guide now also has a section on implementing the naming generation for resources that do not currently have it.

dthvt commented 4 years ago

Is there any plan on when this might be addressed? Are delays just due to lack of manpower for implementation or are there objections internally @ Hashicorp to this proposal? Thx!

max-rocket-internet commented 2 years ago

Any update on this old issue? I got bitten by it today in https://github.com/hashicorp/terraform-provider-aws/issues/25634

geoL86 commented 1 year ago

ooops, it was suprise for me, got bitten as well

ewbankkit commented 1 year ago

Most of the cases of resource with _prefix attributes not supporting resource import have been taken care of as part of other enhancement or refactoring PRs. Some do remain (mainly those where the default prefix of terraform- is replaced with another value, often tf-):

% grep PrefixedUniqueId internal/service/*/*.go
internal/service/acm/certificate.go:            IdempotencyToken: aws.String(id.PrefixedUniqueId("tf")), // 32 character limit
internal/service/autoscaling/attachment.go: d.SetId(id.PrefixedUniqueId(fmt.Sprintf("%s-", asgName)))
internal/service/cloudfront/public_key.go:      d.Set("name", id.PrefixedUniqueId(v.(string)))
internal/service/cloudfront/public_key.go:      d.Set("name", id.PrefixedUniqueId("tf-"))
internal/service/docdb/cluster.go:      identifier = id.PrefixedUniqueId(v.(string))
internal/service/docdb/cluster.go:      identifier = id.PrefixedUniqueId("tf-")
internal/service/docdb/cluster_instance.go:         input.DBInstanceIdentifier = aws.String(id.PrefixedUniqueId(v.(string)))
internal/service/docdb/cluster_instance.go:         input.DBInstanceIdentifier = aws.String(id.PrefixedUniqueId("tf-"))
internal/service/docdb/cluster_parameter_group.go:      groupName = id.PrefixedUniqueId(v.(string))
internal/service/docdb/subnet_group.go:     groupName = id.PrefixedUniqueId(v.(string))
internal/service/elastictranscoder/pipeline.go:     name := id.PrefixedUniqueId("tf-et-")
internal/service/elastictranscoder/preset.go:       name := id.PrefixedUniqueId("tf-et-preset-")
internal/service/elb/attachment.go: d.SetId(id.PrefixedUniqueId(fmt.Sprintf("%s-", elbName)))
internal/service/elb/load_balancer.go:          elbName = id.PrefixedUniqueId(v.(string))
internal/service/elb/load_balancer.go:          elbName = id.PrefixedUniqueId("tf-lb-")
internal/service/elbv2/load_balancer.go:        name = id.PrefixedUniqueId(v.(string))
internal/service/elbv2/load_balancer.go:        name = id.PrefixedUniqueId("tf-lb-")
internal/service/elbv2/target_group.go:     groupName = id.PrefixedUniqueId(v.(string))
internal/service/elbv2/target_group.go:     groupName = id.PrefixedUniqueId("tf-")
internal/service/elbv2/target_group_attachment.go:  d.SetId(id.PrefixedUniqueId(fmt.Sprintf("%s-", d.Get("target_group_arn"))))
internal/service/emr/security_configuration.go:         emrSCName = id.PrefixedUniqueId(v.(string))
internal/service/emr/security_configuration.go:         emrSCName = id.PrefixedUniqueId("tf-emr-sc-")
internal/service/iam/group_policy.go:       policyName = id.PrefixedUniqueId(v.(string))
internal/service/iam/group_policy_attachment.go:    d.SetId(id.PrefixedUniqueId(fmt.Sprintf("%s-", group)))
internal/service/iam/group_policy_attachment_test.go:               // We do not have a way to align IDs since the Create function uses id.PrefixedUniqueId()
internal/service/iam/role_policy.go:        policyName = id.PrefixedUniqueId(v.(string))
internal/service/iam/role_policy_attachment.go: d.SetId(id.PrefixedUniqueId(fmt.Sprintf("%s-", role)))
internal/service/iam/role_policy_attachment_test.go:                // We do not have a way to align IDs since the Create function uses id.PrefixedUniqueId()
internal/service/iam/user_policy.go:        policyName = id.PrefixedUniqueId(v.(string))
internal/service/iam/user_policy_attachment.go: d.SetId(id.PrefixedUniqueId(fmt.Sprintf("%s-", user)))
internal/service/iam/user_policy_attachment_test.go:                // We do not have a way to align IDs since the Create function uses id.PrefixedUniqueId()
internal/service/lightsail/key_pair.go:     kName = id.PrefixedUniqueId(v.(string))
internal/service/mq/broker.go:      CreatorRequestId:        aws.String(id.PrefixedUniqueId(fmt.Sprintf("tf-%s", name))),
internal/service/neptune/cluster.go:        clusterID = id.PrefixedUniqueId(v.(string))
internal/service/neptune/cluster.go:        clusterID = id.PrefixedUniqueId("tf-")
internal/service/neptune/cluster_instance.go:       instanceID = id.PrefixedUniqueId(v.(string))
internal/service/neptune/cluster_instance.go:       instanceID = id.PrefixedUniqueId("tf-")
internal/service/neptune/cluster_parameter_group.go:        groupName = id.PrefixedUniqueId(v.(string))
internal/service/neptune/event_subscription.go:     d.Set("name", id.PrefixedUniqueId(v.(string)))
internal/service/neptune/event_subscription.go:     d.Set("name", id.PrefixedUniqueId("tf-"))
internal/service/pinpoint/app.go:       name = id.PrefixedUniqueId(v.(string))
internal/service/rds/cluster.go:        identifier = id.PrefixedUniqueId(v.(string))
internal/service/rds/cluster.go:        identifier = id.PrefixedUniqueId("tf-")
internal/service/rds/cluster_instance.go:           identifier = id.PrefixedUniqueId(v.(string))
internal/service/rds/cluster_instance.go:           identifier = id.PrefixedUniqueId("tf-")
internal/service/rds/option_group.go:       groupName = id.PrefixedUniqueId(v.(string))
internal/service/route53resolver/endpoint.go:       CreatorRequestId: aws.String(id.PrefixedUniqueId("tf-r53-resolver-endpoint-")),
internal/service/route53resolver/firewall_domain_list.go:       CreatorRequestId: aws.String(id.PrefixedUniqueId("tf-r53-resolver-firewall-domain-list-")),
internal/service/route53resolver/firewall_rule.go:      CreatorRequestId:     aws.String(id.PrefixedUniqueId("tf-r53-resolver-firewall-rule-")),
internal/service/route53resolver/firewall_rule_group.go:        CreatorRequestId: aws.String(id.PrefixedUniqueId("tf-r53-resolver-firewall-rule-group-")),
internal/service/route53resolver/firewall_rule_group_association.go:        CreatorRequestId:    aws.String(id.PrefixedUniqueId("tf-r53-rslvr-frgassoc-")),
internal/service/route53resolver/query_log_config.go:       CreatorRequestId: aws.String(id.PrefixedUniqueId("tf-r53-resolver-query-log-config-")),
internal/service/route53resolver/rule.go:       CreatorRequestId: aws.String(id.PrefixedUniqueId("tf-r53-resolver-rule-")),
internal/service/s3/bucket.go:          rule.ID = aws.String(id.PrefixedUniqueId("tf-s3-lifecycle-"))
internal/service/s3/bucket_notification.go:         lc.Id = aws.String(id.PrefixedUniqueId("tf-s3-lambda-"))
internal/service/s3/bucket_notification.go:         qc.Id = aws.String(id.PrefixedUniqueId("tf-s3-queue-"))
internal/service/s3/bucket_notification.go:         tc.Id = aws.String(id.PrefixedUniqueId("tf-s3-topic-"))

% grep ImportStateVerifyIgnore internal/service/*/*_test.go | grep _prefix
internal/service/docdb/cluster_parameter_group_test.go:             ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/docdb/subnet_group_test.go:                ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/iam/group_policy_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/iam/openid_connect_provider_test.go:               ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/iam/policy_test.go:                ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/iam/role_policy_test.go:               ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/iam/user_policy_test.go:               ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/logs/group_test.go:                ImportStateVerifyIgnore: []string{"retention_in_days", "skip_destroy", "name_prefix"},
internal/service/neptune/cluster_parameter_group_test.go:               ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/neptune/cluster_parameter_group_test.go:               ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/neptune/event_subscription_test.go:                ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix", "option"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix", "option"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/rds/option_group_test.go:              ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/s3/bucket_test.go:             ImportStateVerifyIgnore: []string{"force_destroy", "bucket_prefix"},
internal/service/s3/bucket_test.go:             ImportStateVerifyIgnore: []string{"force_destroy", "bucket_prefix"},
internal/service/signer/signing_profile_permission_test.go:             ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/signer/signing_profile_permission_test.go:             ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/signer/signing_profile_permission_test.go:             ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/signer/signing_profile_permission_test.go:             ImportStateVerifyIgnore: []string{"name_prefix"},
internal/service/signer/signing_profile_test.go:                ImportStateVerifyIgnore: []string{"name_prefix"},

ewbankkit commented 1 year ago

PRs/Issues to look at:

github-actions[bot] commented 1 year ago

This functionality has been released in v5.22.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions[bot] commented 12 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws