[Bug]: AWS App Runner services get stuck in state OPERATION_IN_PROGRESS

Terraform Core Version

1.9.2

AWS Provider Version

5.58.0

Affected Resource(s)

aws_apprunner_service
aws_elasticache_cluster

Expected Behavior

If I change the content of a variable and apply not other changes, a 'terraform apply' should rebuild an existing App Runner services created earlier by the same Terraform configuration.

Actual Behavior

The Terrafrom command times out after 20 minutes and the service created earlier is not updated but stuck in the state 'OPERATION_IN_PROGRESS' and cannot be deleted by Terraform, AWS cli or AWS console. Problem is I will still get billed for the services, but cannot remove them.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_apprunner_service" "test-service" {
  service_name = var.instance_name

  health_check_configuration {
      timeout = 3
      interval = 10
      path = var.health_check_url
      protocol = "HTTP"
  }

  source_configuration {
    authentication_configuration {
      connection_arn = var.connection_arn
    }

    code_repository {
      code_configuration {
        code_configuration_values {
          port          = "8000"
          runtime       = "PYTHON_311"
          runtime_environment_variables = {
        # Only change before trying to apply
            REDIS_URL =  one(aws_elasticache_cluster.rediscluster.cache_nodes).address
            MY_APP_PORT = "8000"
          }
          start_command = "./start.sh"
        }
        configuration_source = "API"
      }

      repository_url = var.repository_url
      source_code_version {
        type  = "BRANCH"
        value = var.repository_branch
      }
    }

     auto_deployments_enabled = false
  }

  instance_configuration {
      instance_role_arn = var.instance_role_arn
    } 

  network_configuration {
    egress_configuration{
        egress_type = "DEFAULT"
    }

    ingress_configuration{
      is_publicly_accessible = true
    }
  }

  tags = {
    Name = var.instance_name
  }
}

resource "aws_apprunner_vpc_connector" "connector" {
  vpc_connector_name = "apprunner_vpc_connector"
  subnets            = aws_subnet.default.*.id
  security_groups    = aws_security_group.default.*.id
}

Steps to Reproduce

Use Terraform to create App Runner and connected services with 'terraform apply'
Only change something in the configuration of the App Runner service, not in the other services or the application code
Run 'terraform apply' again

Debug Output

$ terraform apply

aws_apprunner_service.test-service: Modifying... [id=arn:aws:apprunner:eu-central-1:xxxxxxxxx:service/aws_redis_cluster_test/xxxxxxxxx]
aws_apprunner_service.test-service: Still modifying... [id=arn:aws:apprunner:eu-central-1:xxxxxxxxx..._test/xxxxxxxxx, 10s elapsed]
aws_apprunner_service.test-service: Still modifying... [id=arn:aws:apprunner:eu-central-1:xxxxxxxxx..._test/xxxxxxxxx, 20s elapsed]
(...)
aws_apprunner_service.test-service: Still modifying... [id=arn:aws:apprunner:eu-central-1:xxxxxxxxx..._test/xxxxxxxxx, 21m18s elapsed]
│ Error: waiting for App Runner Service (arn:aws:apprunner:eu-central-1:xxxxxx:service/aws_redis_cluster_test/xxxxxxxxx) update: timeout while waiting for state to become 'RUNNING' (last state: 'OPERATION_IN_PROGRESS', timeout: 20m0s)
│
│   with aws_apprunner_service.test-service,
│   on main.tf line 2, in resource "aws_apprunner_service" "test-service":
│    2: resource "aws_apprunner_service" "test-service" {
│

$ terraform destroy

2024-07-15 10:44:44,861 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/v2/2.17.5/dist/awscli/botocore/cacert.pem
2024-07-15 10:44:44,861 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): apprunner.eu-central-1.amazonaws.com:443
2024-07-15 10:44:44,968 - MainThread - urllib3.connectionpool - DEBUG - https://apprunner.eu-central-1.amazonaws.com:443 "POST / HTTP/1.1" 400 141
2024-07-15 10:44:44,969 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Mon, 15 Jul 2024 08:44:45 GMT', 'Content-Type': 'application/x-amz-json-1.0', 'Content-Length': '141', 'Connection': 'keep-alive', 'x-amzn-RequestId': 'xxxxxx'}
2024-07-15 10:44:44,970 - MainThread - botocore.parsers - DEBUG - Response body:
b'{"__type":"com.amazonaws.apprunner#InvalidStateException","Message":"Service cannot be deleted in the current state: OPERATION_IN_PROGRESS."}'
2024-07-15 10:44:44,972 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Mon, 15 Jul 2024 08:44:45 GMT', 'Content-Type': 'application/x-amz-json-1.0', 'Content-Length': '141', 'Connection': 'keep-alive', 'x-amzn-RequestId': 'xxxxxx'}
2024-07-15 10:44:44,972 - MainThread - botocore.parsers - DEBUG - Response body:
b'{"__type":"com.amazonaws.apprunner#InvalidStateException","Message":"Service cannot be deleted in the current state: OPERATION_IN_PROGRESS."}'
2024-07-15 10:44:44,973 - MainThread - botocore.hooks - DEBUG - Event needs-retry.apprunner.DeleteService: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x7fb2cf33fd10>>
2024-07-15 10:44:44,973 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2024-07-15 10:44:44,973 - MainThread - botocore.hooks - DEBUG - Event after-call.apprunner.DeleteService: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x7fb2cf33f790>>
2024-07-15 10:44:44,974 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli/clidriver.py", line 499, in main
  File "awscli/clidriver.py", line 634, in __call__
  File "awscli/clidriver.py", line 837, in __call__
  File "awscli/clidriver.py", line 963, in invoke
  File "awscli/clidriver.py", line 975, in _make_client_call
  File "awscli/botocore/client.py", line 360, in _api_call
  File "awscli/botocore/client.py", line 739, in _make_api_call
botocore.errorfactory.InvalidStateException: An error occurred (InvalidStateException) when calling the DeleteService operation: Service cannot be deleted in the current state: OPERATION_IN_PROGRESS.

An error occurred (InvalidStateException) when calling the DeleteService operation: Service cannot be deleted in the current state: OPERATION_IN_PROGRESS.

Panic Output

No response

Important Factoids

No response

References

No response

hashicorp / terraform-provider-aws