hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.62k stars 9.01k forks source link

Add "status" to aws_ec2_host attributes #23402

Open BrianMusson opened 2 years ago

BrianMusson commented 2 years ago

Community Note

Description

Mac instances in particular will put a dedicated host in 'pending' status when stopping or terminating a macOS instance. aws_instance does not have the logic to wait for the instance to become 'available'. Please add a status output so that we can use depend_on on the aws_instance resource.

New or Affected Resource(s)

Potential Terraform Configuration

resource "aws_instance" "macOS" {
  depends_on = [aws_ec2_host.dedicated]
}

Ideally we would wait until the dedicated host was "available" before attempting to provision an instance.

References

mikegyver99 commented 2 years ago

Hope I am not adding noise as we are experiencing this issue as well: terraform -replace="aws_instance.<name>" command. After instance is terminated, the aws_ec2_host goes immediately into pending state. So when terraform attempts to re-deploy the aws_instance, (as part of -replace workflow) it returns an error "InvalidHostState". It generally takes about 30-40 minutes for the aws_ec2_host to change state to "available" after aws_instance termination.

image

pending | The Dedicated Host cannot be used for new instance launches. It is either being modified to support multiple instance types, or a host recovery is in progress. -- | --

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-dedicated-hosts-work.html

InvalidHostState | The Dedicated Host must be in the available state to complete the operation. -- | --

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html

hakuno commented 5 months ago

It worked without depends_on meta-argument.

But, be sure there has host_id set up properly.

chrisdenton-ct commented 5 months ago

I think this is still an issue, as of February 2024. Dedicated hosts in the mac2 instance family can spend up to two hours in the Pending state.

When I update the AMI in terraform and run an "apply" I get an error as soon as the old instance is successfully terminated and the underlying dedicated host has gone into Pending state:

Error: creating EC2 Instance: InsufficientCapacityOnHost: Dedicated host h-xxxxxxxxxxxxxxx has insufficient capacity to launch the instances in this request.

Ideally, terraform would wait for the dedicated host to become Available, and then bring up the new instance. As it is, I have to watch for this in the AWS console, and then re-run my apply once the host is ready.

hakuno commented 5 months ago

Ideally, terraform would wait for the dedicated host to become Available

It isn't an issue. Not a Terraform issue. It's an async event of AWS EC2 bare metal instances.

image

So we can't expect some "timeout-less" apply.

Ps. Even when the dedicated host isn't in pending status anymore, it sill takes a time to be able to run/allocate the instance again.

Ps. If you need to reconcile that automatically, you can try some GitOps approach like FluxCD to run Terraform and solve drifts as well.

chrisdenton-ct commented 5 months ago

Thanks for the reply, @hakuno!

It isn't an issue. Not a Terraform issue. It's an async event of AWS EC2 bare metal instances.

It's certainly an issue I experience through terraform. I guess whether it should be fixed in the aws provider, or not, is in some ways a philosophical question.

Ps. If you need to reconcile that automatically, you can try some GitOps approach like FluxCD to run Terraform and solve drifts as well.

It doesn't need to be automatic for me, thankfully, as that seems like quite an elaborate workaround.