search

canonical / observability-libs

A collection of charm libraries curated by the Observability team.
https://charmhub.io/observability-libs
Apache License 2.0
3 stars 8 forks source link

Fix traefik lb being constantly re-created on controller restart #104

Closed IbraAoad closed 2 months ago

IbraAoad commented 2 months ago

Issue

Traefik k8s service is recreated on each controller restart, this causes the k8s api to request a new external IP for the LB when a pool of IPs exist

Solution

Call the _remove_service only when the on_remove hook is observed instead of calling it on each stop operation, also adjust the upgrade logic to delete any previous LB if service type is changed to ClusterIP

Context

From Managed Solutions

I am facing some issues with Traefik on top of OpenStack cloud. I see traefik now create an additional service called traefik-lb correctly b/c of #319 but any juju controller restart causes it to be recreated, which subsequently triggers a creation of a new LB in OpenStack Octavia and my LB Floating IP changes, but i have a lot of configuration around that LB (FIP, DNS, lxd loki addres and firewall) that makes COS unusable

Testing Instructions

  1. juju deploy traefik
  2. juju deploy catalogue
  3. juju relate traefik catalogue
  4. kubectl delete pod controller-0 -n controller-k8s (wait until it's completely deleted and comes back up again)
  5. kubectl get svc -A -o wide (observe that the traefik lb isn't restarted)

Tandem PR: https://github.com/canonical/traefik-k8s-operator/pull/381