Describe the bug
When I try to deploy backstage pattern, the eks-loadbalancer resource creation is timing out with errors like
Received response status [FAILED] from custom resource. Message returned: TimeoutError: {"state":"TIMEOUT","reason":"Waiter has timed out"} at checkExceptions (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/util-waiter/dist-cjs/index.js:59:26) at waitUntilFunctionActiveV2 (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:5826:49) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async defaultInvokeFunction (/var/task/outbound.js:1:1024) at async invokeUserFunction (/var/task/framework.js:1:2287) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1676) (RequestId: 16ad0de3-1e44-438b-814c-ac208323c802)
On searching for the issue in cloudwatch logs, I get this
2024-07-26T16:49:08.745Z 885eae61-9575-48f9-84d1-c2058a66a94f INFO [provider-framework] executing user function arn:aws:lambda:ap-south-1:323155100510:function:backstage-blueprint-awscdkawseksKu-Handler886CB40B-TCMBwnyiTMag with payload {
"RequestType": "Create",
"ServiceToken": "arn:aws:lambda:ap-south-1:323155100510:function:backstage-blueprint-awscd-ProviderframeworkonEvent-8xHpBQWOgq7v",
"ResponseURL": "...",
"StackId": "arn:aws:cloudformation:ap-south-1:323155100510:stack/backstage-blueprint/faf29dd0-4b6c-11ef-83b5-027ab549ec09",
"RequestId": "16ad0de3-1e44-438b-814c-ac208323c802",
"LogicalResourceId": "backstageblueprintawsloadbalancercontrollermanifestawsloadbalancercontrollerServiceAccountResource42E6EEE9",
"ResourceType": "Custom::AWSCDK-EKS-KubernetesResource",
"ResourceProperties": {
"ServiceToken": "arn:aws:lambda:ap-south-1:323155100510:function:backstage-blueprint-awscd-ProviderframeworkonEvent-8xHpBQWOgq7v",
"PruneLabel": "aws.cdk.eks/prune-c8e52c43061de408d9284c707a9eaa0c6f9cfaef0f",
"ClusterName": "backstage-blueprint",
"Manifest": "[{\"apiVersion\":\"v1\",\"kind\":\"ServiceAccount\",\"metadata\":{\"name\":\"aws-load-balancer-controller\",\"namespace\":\"kube-system\",\"labels\":{\"aws.cdk.eks/prune-c8e52c43061de408d9284c707a9eaa0c6f9cfaef0f\":\"\",\"app.kubernetes.io/name\":\"aws-load-balancer-controller\"},\"annotations\":{\"eks.amazonaws.com/role-arn\":\"arn:aws:iam::323155100510:role/backstage-blueprint-backstageblueprintawsloadbalanc-bYL4sAgx0WEu\"}}}]",
"RoleArn": "arn:aws:iam::323155100510:role/backstage-blueprint-backstageblueprintCreationRoleE-sYYVkymowvO8"
}
}
2024-07-26T16:52:23.572Z 885eae61-9575-48f9-84d1-c2058a66a94f INFO [provider-framework] CREATE failed, responding with a marker physical resource id so that the subsequent DELETE will be ignored
2024-07-26T16:52:23.603Z 885eae61-9575-48f9-84d1-c2058a66a94f INFO [provider-framework] submit response to cloudformation https://cloudformation-custom-resource-response-apsouth1.s3.ap-south-1.amazonaws.com//arn%3Aaws%3Acloudformation%3Aap-south-1%3A323155100510%3Astack/backstage-blueprint/faf29dd0-4b6c-11ef-83b5-027ab549ec09%7CbackstageblueprintawsloadbalancercontrollermanifestawsloadbalancercontrollerServiceAccountResource42E6EEE9%7C16ad0de3-1e44-438b-814c-ac208323c802?*** {
"Status": "FAILED",
"Reason": "TimeoutError: {\"state\":\"TIMEOUT\",\"reason\":\"Waiter has timed out\"}\n at checkExceptions (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/util-waiter/dist-cjs/index.js:59:26)\n at waitUntilFunctionActiveV2 (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:5826:49)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async defaultInvokeFunction (/var/task/outbound.js:1:1024)\n at async invokeUserFunction (/var/task/framework.js:1:2287)\n at async onEvent (/var/task/framework.js:1:369)\n at async Runtime.handler (/var/task/cfn-response.js:1:1676)",
"StackId": "arn:aws:cloudformation:ap-south-1:323155100510:stack/backstage-blueprint/faf29dd0-4b6c-11ef-83b5-027ab549ec09",
"RequestId": "16ad0de3-1e44-438b-814c-ac208323c802",
"PhysicalResourceId": "AWSCDK::CustomResourceProviderFramework::CREATE_FAILED",
"LogicalResourceId": "backstageblueprintawsloadbalancercontrollermanifestawsloadbalancercontrollerServiceAccountResource42E6EEE9"
}
Describe the bug When I try to deploy backstage pattern, the eks-loadbalancer resource creation is timing out with errors like
On searching for the issue in cloudwatch logs, I get this
Lambda is timing out I think
To Reproduce
cc: @freschri