[Bug]: IncorrectSpotRequestState when restarting Spot Instance to apply user_data changes

Terraform Core Version

1.5.3

AWS Provider Version

5.9.0

Affected Resource(s)

aws_instance

Expected Behavior

When user_data or user_data_base64of a spot instance is changed (or in general, actions that force a spot EC2 instance to be stopped and restarted), Terraform should wait + retry on IncorrectSpotRequestRate until the instance can be started.

Actual Behavior

The spot instance is stopped, but fail to be restarted. This is because there's a delay for the persistent spot request to be updated from state: active, status: marked-for-stop --> state: disabled, status: instance-stopped-by-user.

When Terraform tries to start the instance again, the request is still active => the instance can't be started.

When the request has transitioned to disabled, the instance can be started.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_instance" "example_instance" {
  ami                   = "some-ami"
  instance_type = "t2.micro"

  instance_market_options {
    spot_options {
      instance_interruption_behaviour = "stop"
      spot_instance_type = "persistent"
    }
  }

  user_data = <<-EOF
    #!/bin/bash
    echo "Hello, World!"
  EOF

  user_data_replace_on_change = false
}

Steps to Reproduce

Create a spot aws_instance resource with some user_data or user_data_base64. spot_instance_type needs to be set to persistent (see config file above), otherwise the request is closed after the instance is launched and won't reflect the bug.
terraform apply to create the instance
Update user_data to something else
terraform apply to trigger the stop and restart
Observe the failure

Debug Output

No response

Panic Output

Error: updating EC2 Instance (i-0942bf2c1f1a5bbbd) user data base64: starting EC2 Instance (i-0942bf2c1f1a5bbbd): IncorrectSpotRequestState: You can't start the Spot Instance 'i-0942bf2c1f1a5bbbd' because the associated Spot Instance request is not in an appropriate state to support start.
status code: 400, request id: 382cbde9-1f5c-48f5-b001-f0722e6fee58
on main.tf line 225, in resource "aws_instance" "workspace":
225: resource "aws_instance" "workspace" {

Important Factoids

No response

References

No response

Would you like to implement a fix?

Yes

hashicorp / terraform-provider-aws