terraform import after failed terraform apply because of already-existing resources

marekaf commented 5 years ago

Current Terraform Version

Terraform v0.11.11
+ provider.aws v1.58.0

Use-cases

when I run terraform apply and I get "resource X already exists"

Error: Error applying plan:

4 error(s) occurred:

* aws_iam_role.nodes-kops-k8s-local: 1 error(s) occurred:

* aws_iam_role.nodes-kops-k8s-local: Error creating IAM Role nodes.kops.k8s.local: EntityAlreadyExists: Role with name nodes.kops.k8s.local already exists.
    status code: 409, request id: d2c0f05b-304f-11e9-bcd8-7fc99914b1a7

it is probably it was created outside of my current terraform state (manually, by other tool than terraform or by different terraform "session")

what I usually want is to terraform import the already-existing resource to have it in my terraform state (I obviously don't want to recreate them, especially key rings)

Attempted Solutions

so I'd manually run:

terraform import aws_iam_role.nodes-kops-k8s-local nodes.kops.k8s.local

for all the already-existing resources (which can be very tedious)

Proposal

what if a failed terraform apply would propose creating several terraform import commands for use to copy/paste and execute?

btw I love terraform, thank you for all the hard work <3

ch3ck commented 5 years ago

Had exactly this same issue today. Wonder if there's any updates to this in v0.11.13

kenorb commented 5 years ago

Same here with the v0.11.11 and the latest Terraform v0.11.13.

Error: Error applying plan:

15 error(s) occurred:

* aws_key_pair.myproject: 1 error(s) occurred:

* aws_key_pair.myproject: Error import KeyPair: InvalidKeyPair.Duplicate: The keypair 'ec2-user-docker-cluster-host' already exists.
  status code: 400, request id: 958d383f-707d-48fa-b13c-45ffeafcf4bc
* aws_ecr_repository.myproject: 1 error(s) occurred:

* aws_ecr_repository.myproject: error creating ECR repository: RepositoryAlreadyExistsException: The repository with name 'myproject' already exists in the registry with id '896267410871'
  status code: 400, request id: 60e6733b-4a82-11e9-b06e-b3fcb14e31db
* aws_security_group.myproject_database: 1 error(s) occurred:

* aws_security_group.myproject_database: Error creating Security Group: InvalidGroup.Duplicate: The security group 'myproject_database' already exists for VPC 'vpc-dd1571bb'
  status code: 400, request id: 7ffd779d-e91a-4ae2-80be-6f1976e533c1
* module.notify_slack.aws_lambda_function.notify_slack: 1 error(s) occurred:

* aws_lambda_function.notify_slack: Error creating Lambda function: ResourceConflictException: Function already exist: myproject_alerts_notify_slack
  status code: 409, request id: 61a5a3eb-4a82-11e9-a1de-1ba8f1ea4b05
* aws_db_parameter_group.myproject_database: 1 error(s) occurred:

* aws_db_parameter_group.myproject_database: Error creating DB Parameter Group: DBParameterGroupAlreadyExists: Parameter group myproject already exists
  status code: 400, request id: 59defa4d-e8c2-4b8f-8c8e-c6fcbea790de
* aws_rds_cluster_parameter_group.myproject_database: 1 error(s) occurred:

* aws_rds_cluster_parameter_group.myproject_database: Error creating DB Cluster Parameter Group: DBParameterGroupAlreadyExists: Parameter group myproject already exists
  status code: 400, request id: 9766861a-f5d4-4a1c-bbea-c9bfc804ccbb
* aws_iam_role.myproject_app: 1 error(s) occurred:

* aws_iam_role.myproject_app: Error creating IAM Role myproject-app: EntityAlreadyExists: Role with name myproject-app already exists.
  status code: 409, request id: 61165cc4-4a82-11e9-abc1-b9f2dc0df61e
* aws_iam_role.database-admin: 1 error(s) occurred:

* aws_iam_role.database-admin: Error creating IAM Role DatabaseAdmin: EntityAlreadyExists: Role with name DatabaseAdmin already exists.
  status code: 409, request id: 610c2362-4a82-11e9-abc1-b9f2dc0df61e
* aws_security_group.app_lb: 1 error(s) occurred:

* aws_security_group.app_lb: Error creating Security Group: InvalidGroup.Duplicate: The security group 'myproject-app-lb' already exists for VPC 'vpc-dd1571bb'
  status code: 400, request id: 8a124bdd-5457-45ea-825a-611448a69c69
* aws_appautoscaling_target.target: 1 error(s) occurred:

* aws_appautoscaling_target.target: Error creating application autoscaling target: ValidationException: ECS service doesn't exist: service/myproject/myproject_app
  status code: 400, request id: 60e3b40c-4a82-11e9-bfd5-ad3c3f26773a
* module.myproject-ecs-cluster.aws_iam_policy.logging: 1 error(s) occurred:

* aws_iam_policy.logging: Error creating IAM policy myproject-staging-ecs-cluster-logging: EntityAlreadyExists: A policy called myproject-staging-ecs-cluster-logging already exists. Duplicate names are not allowed.
  status code: 409, request id: 615bc99b-4a82-11e9-8034-c725489e4396
* module.myproject-ecs-cluster.aws_security_group.instance: 1 error(s) occurred:

* aws_security_group.instance: Error creating Security Group: InvalidGroup.Duplicate: The security group 'myproject-staging-ecs-cluster' already exists for VPC 'vpc-dd1571bb'
  status code: 400, request id: b4354b11-ce92-4869-bfbc-baed88bf0a66
* aws_iam_role.ecs_events: 1 error(s) occurred:

* aws_iam_role.ecs_events: Error creating IAM Role ecs_events: EntityAlreadyExists: Role with name ecs_events already exists.
  status code: 409, request id: 611746a2-4a82-11e9-93ee-89a3d4b4209c
* module.myproject-ecs-cluster.aws_iam_role.instance: 1 error(s) occurred:

* aws_iam_role.instance: Error creating IAM Role myproject-staging-ecs-cluster-role: EntityAlreadyExists: Role with name myproject-staging-ecs-cluster-role already exists.
  status code: 409, request id: 6149c8c2-4a82-11e9-b2d6-db3c8c5385b6
* module.myproject-ecs-cluster.aws_placement_group.instance: 1 error(s) occurred:

* aws_placement_group.instance: InvalidPlacementGroup.Duplicate: The placement group 'myproject-staging-ecs-cluster-pg' already exists.
  status code: 400, request id: b8910c0a-550c-420a-983e-9f6f5005ff50

Removing them or importing manually it's a mess.

Maybe -force or -import option should be added?

Ben197 commented 4 years ago

Sorry guys this is really a trade off, if you want to deliver tools for an enterprise you should solve your issues !

Ben197 commented 4 years ago

Spent almost a day on this issue :-(

marekaf commented 4 years ago

Sorry guys this is really a trade off, if you want to deliver tools for an enterprise you should solve your issues !

this is open source, get an enterprise terraform cloud support if you have these expectations :)

infogulch commented 3 years ago

At the least it should print out both the 'resource id' and 'resource address' when displaying the error; this would greatly reduce the friction to importing (since you're not reduced to just guessing the address). That said, it seems this would be a provider-specific fix, because each provider does its own error handling.

For example, in the azure provider the function that prints the error is ImportAsExistsError(resourceName, id string) error and prints an error that looks like "A resource with the ID %q already exists - to be managed via Terraform this resource needs to be imported into the State.". But this function doesn't have access to the resource address and I'm not sure that the many callers have easy access to the resource address either. Maybe there's a non-obvious way to get the resource address from a pluginsdk.ResourceData?

This would be a great improvement in dealing with this issue, but "fixing" it would require adjusting handling of this error on every resource in each provider individually.

chessai commented 2 years ago

Running into this now, but with already existing resources which were created by terraform. For organisational purposes, I separate code out into modules in their own directories, e.g. modules/{foo,bar}. Then in the top level main.tf I just:

module "foo" {
  source = "./modules/foo"
}

But this will invalidate all of the previously created resources if I defined them in the top level main.tf first, simply because they moved to a new directory.

EDIT: Terraform would really benefit from content addressing.

hashicorp / terraform