Closed meyerkev closed 1 year ago
Describe the bug Our healthchecks only pass when using HTTP1 and our Target group is always HTTP2 even when we set it explicitly.
Specifically, we're running thanos-query as part of the thanos chart so we don't control their compatibility.
Steps to reproduce
Create an ingress with this particular set of annotations
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: alb.ingress.kubernetes.io/backend-protocol: HTTP alb.ingress.kubernetes.io/backend-protocol-version: HTTP1
Apply the ingress
Get these results
Expected outcome I should get an HTTP/1 target group that checks, what, in my case, was port 9090 on / every 5 seconds.
Instead, I get an HTTP/2 target group instead.
Environment
Additional Context: I did once manage to get it to work and make me an HTTP/1 version and it did in fact briefly work.
The full ingress definition:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: alb.ingress.kubernetes.io/backend-protocol: HTTP alb.ingress.kubernetes.io/backend-protocol-version: HTTP1 alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:795586375822:certificate/83453e70-53c2-4e5f-be34-e55b15b3af22 alb.ingress.kubernetes.io/group.name: usw2-staging-internal alb.ingress.kubernetes.io/group.order: "106" alb.ingress.kubernetes.io/healthcheck-port: "9090" alb.ingress.kubernetes.io/inbound-cidrs: 10.0.0.0/8 alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internal alb.ingress.kubernetes.io/ssl-redirect: "443" alb.ingress.kubernetes.io/success-codes: 200,302,301,304 alb.ingress.kubernetes.io/target-type: instance external-dns.alpha.kubernetes.io/alias: "true" external-dns.alpha.kubernetes.io/hostname: thanos-query.usw2-staging.mywebsite.io external-dns.alpha.kubernetes.io/ttl: "60" kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{"alb.ingress.kubernetes.io/backend-protocol":"HTTP","alb.ingress.kubernetes.io/backend-protocol-version":"HTTP1","alb.ingress.kubernetes.io/certificate-arn":"arn:aws:acm:us-west-2:795586375822:certificate/83453e70-53c2-4e5f-be34-e55b15b3af22","alb.ingress.kubernetes.io/group.name":"usw2-staging-internal","alb.ingress.kubernetes.io/group.order":"106","alb.ingress.kubernetes.io/healthcheck-port":"9090","alb.ingress.kubernetes.io/inbound-cidrs":"10.0.0.0/8","alb.ingress.kubernetes.io/listen-ports":"[{\"HTTP\": 80}, {\"HTTPS\":443}]","alb.ingress.kubernetes.io/scheme":"internal","alb.ingress.kubernetes.io/ssl-redirect":"443","alb.ingress.kubernetes.io/success-codes":"200,302,301","alb.ingress.kubernetes.io/target-type":"instance","external-dns.alpha.kubernetes.io/alias":"true","external-dns.alpha.kubernetes.io/hostname":"thanos-query.usw2-staging.mywebsite.io","external-dns.alpha.kubernetes.io/ttl":"60","kubernetes.io/ingress.class":"alb"},"name":"thanos-query","namespace":"monitoring"},"spec":{"rules":[{"host":"thanos-query.usw2-staging.mywebsite.io","http":{"paths":[{"backend":{"service":{"name":"thanos-query","port":{"number":10901}}},"pathType":"ImplementationSpecific"}]}}]}} kubernetes.io/ingress.class: alb creationTimestamp: "2023-04-18T00:36:56Z" finalizers: - group.ingress.k8s.aws/usw2-staging-internal generation: 1 name: thanos-query namespace: monitoring resourceVersion: "159293350" uid: 8e14979d-d0d4-4af6-a1c6-695ed1bd47dd spec: rules: - host: thanos-query.usw2-staging.mywebsite.io http: paths: - backend: service: name: thanos-query port: number: 10901 pathType: ImplementationSpecific status: loadBalancer: ingress: - hostname: internal-k8s-usw2stagingintern-84b72ea72c-450260535.us-west-2.elb.amazonaws.com
Never mind me, our service was overriding the ingress with HTTP2
Describe the bug Our healthchecks only pass when using HTTP1 and our Target group is always HTTP2 even when we set it explicitly.
Specifically, we're running thanos-query as part of the thanos chart so we don't control their compatibility.
Steps to reproduce
Create an ingress with this particular set of annotations
Apply the ingress
Get these results
Expected outcome I should get an HTTP/1 target group that checks, what, in my case, was port 9090 on / every 5 seconds.
Instead, I get an HTTP/2 target group instead.
Environment
Additional Context: I did once manage to get it to work and make me an HTTP/1 version and it did in fact briefly work.
The full ingress definition: