aws / aws-application-networking-k8s

A Kubernetes controller for Amazon VPC Lattice
https://www.gateway-api-controller.eks.aws.dev/
Apache License 2.0
175 stars 50 forks source link

Fix flaky e2e test for Access Log Subscriptions #673

Open erikfuller opened 3 weeks ago

erikfuller commented 3 weeks ago

This e2e tests appear to be regularly failing on:

Access Log Policy creation produces Access Log Subscriptions with Bucket, Log Group, and Delivery Stream destinations on the same targetRef

https://github.com/aws/aws-application-networking-k8s/actions/workflows/e2e-test.yaml

Error:

Access Log Policy creation produces Access Log Subscriptions with Bucket, Log Group, and Delivery Stream destinations on the same targetRef
/home/runner/work/aws-application-networking-k8s/aws-application-networking-k8s/test/suites/integration/access_log_policy_test.go:398
{"level":"info","ts":"2024-11-01T18:14:08.365Z","caller":"test/framework.go:186","msg":"Creating objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-bucket"}
{"level":"info","ts":"2024-11-01T18:14:08.469Z","caller":"test/framework.go:186","msg":"Creating objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-log-group"}
{"level":"info","ts":"2024-11-01T18:14:08.561Z","caller":"test/framework.go:186","msg":"Creating objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-delivery-stream"}
  [FAILED] in [It] - /home/runner/work/aws-application-networking-k8s/aws-application-networking-k8s/test/suites/integration/access_log_policy_test.go:468 @ 11/01/24 18:17:08.654
{"level":"info","ts":"2024-11-01T18:17:09.729Z","caller":"test/framework.go:264","msg":"Deleting objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-bucket"}
{"level":"info","ts":"2024-11-01T18:17:09.834Z","caller":"test/framework.go:282","msg":"Waiting for NotFound, objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-bucket"}
{"level":"info","ts":"2024-11-01T18:17:11.018Z","caller":"test/framework.go:264","msg":"Deleting objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-delivery-stream"}
{"level":"info","ts":"2024-11-01T18:17:11.118Z","caller":"test/framework.go:282","msg":"Waiting for NotFound, objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-delivery-stream"}
{"level":"info","ts":"2024-11-01T18:17:16.678Z","caller":"test/framework.go:264","msg":"Deleting objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-log-group"}
{"level":"info","ts":"2024-11-01T18:17:16.781Z","caller":"test/framework.go:282","msg":"Waiting for NotFound, objects: *v1alpha1.AccessLogPolicy/test-access-log-policy-log-group"}
{"level":"info","ts":"2024-11-01T18:17:16.872Z","caller":"test/framework.go:219","msg":"Found 2 route objects"}
{"level":"info","ts":"2024-11-01T18:17:16.964Z","caller":"test/framework.go:238","msg":"Clearing http route rules for foewo-30-fl5n4eonfl"}
{"level":"info","ts":"2024-11-01T18:17:17.160Z","caller":"test/framework.go:251","msg":"Clearing grpc route rules for songe-33-smqlribvn9"}
{"level":"info","ts":"2024-11-01T18:17:27.258Z","caller":"test/framework.go:264","msg":"Deleting objects: *v1.HTTPRoute/foewo-30-fl5n4eonfl, *v1alpha2.GRPCRoute/songe-33-smqlribvn9, *v1.Service/foewo-30-fl5n4eonfl, *v1.Service/hawks-32-xjxmz6cucg, *v1.Deployment/yaksc-29-kp9njt47r0, *v1.Deployment/rippe-31-rdqyucbsbp"}
{"level":"info","ts":"2024-11-01T18:17:27.457Z","caller":"test/framework.go:282","msg":"Waiting for NotFound, objects: *v1.HTTPRoute/foewo-30-fl5n4eonfl, *v1alpha2.GRPCRoute/songe-33-smqlribvn9, *v1.Service/foewo-30-fl5n4eonfl, *v1.Service/hawks-32-xjxmz6cucg, *v1.Deployment/yaksc-29-kp9njt47r0, *v1.Deployment/rippe-31-rdqyucbsbp"}
• [FAILED] [222.599 seconds]
Access Log Policy [It] creation produces Access Log Subscriptions with Bucket, Log Group, and Delivery Stream destinations on the same targetRef
/home/runner/work/aws-application-networking-k8s/aws-application-networking-k8s/test/suites/integration/access_log_policy_test.go:398

  [FAILED] Timed out after 180.001s.
  The function passed to Eventually failed at /home/runner/work/aws-application-networking-k8s/aws-application-networking-k8s/test/suites/integration/access_log_policy_test.go:459 with:
  Expected
      <int>: 2
  to be equivalent to
      <int>: 3
  In [It] at: /home/runner/work/aws-application-networking-k8s/aws-application-networking-k8s/test/suites/integration/access_log_policy_test.go:468 @ 11/01/24 18:17:08.654

Not sure if it just needs more time or if there's something more going on here. I did run the test locally and it passed.