search

pulumi / pulumi-eks

A Pulumi component for easily creating and managing an Amazon EKS Cluster
https://www.pulumi.com/registry/packages/eks/
Apache License 2.0
169 stars 78 forks source link

Fargate tests are flakey #1041

Open rquitales opened 4 months ago

rquitales commented 4 months ago

What happened?

Surfaced in #1039. Our daily cron job fails about 33% of the time due to our Fargate tests.

The error from the test is:

 utils.go:290: 
            Error Trace:    /home/runner/work/pulumi-eks/pulumi-eks/examples/utils/utils.go:290
                                        /home/runner/work/pulumi-eks/pulumi-eks/examples/utils/utils.go:225
                                        /home/runner/work/pulumi-eks/pulumi-eks/examples/utils/utils.go:84
                                        /home/runner/work/pulumi-eks/pulumi-eks/examples/utils/utils.go:90
                                        /opt/hostedtoolcache/go/1.21.6/x64/src/runtime/asm_amd64.s:1650
            Error:          Not equal: 
                            expected: 3
                            actual  : 2
            Test:           TestAccFargate
            Messages:       2 out of 3 Pods are ready

It is unclear why only 2 of the 3 pods can become live and requires further debugging.

Example

https://github.com/pulumi/pulumi-eks/issues/1039

Output of pulumi about

-

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

thomas11 commented 4 months ago

@rquitales I think we also need to disable Fargate tests until this is resolved, otherwise we'll constantly get new P1s like this #1043.