AKS/LB: avoid control plane changes to the LB health probe endpoints

Altinn / altinn-platform

Altinn Platform infrastructure

6 stars 1 forks source link

AKS/LB: avoid control plane changes to the LB health probe endpoints #879

Open sduranc opened 1 month ago

sduranc commented 1 month ago

AKS seems to be changing and updating the LB health endpoint to a defined http
This breaks incoming traffic
Tune the current SLOs to detect this problem or create a new one
Evaluate using an external prober for minimal synthetic traffic

bengtfredh commented 3 weeks ago

I am pretty sure this is done on signals from traefik. If you set externalTrafficPolicy: Local on traefik svc you get a single http health probe. If you set externalTrafficPolicy: Cluster you get 2 tcp health probes.

If you compare platform-at22 which have externalTrafficPolicy: Local it has 1 http health probe, while platform-at23 have externalTrafficPolicy: Cluster and 2 tcp heath probes.