search

adyanth / cloudflare-operator

A Kubernetes Operator to create and manage Cloudflare Tunnels and DNS records for (HTTP/TCP/UDP*) Service Resources
https://adyanth.site/posts/migration-compose-k8s/cloudflare-tunnel-operator-architecture/
Apache License 2.0
343 stars 35 forks source link

cloudflared deployment restarts endlessly #111

Open maxpain opened 1 day ago

maxpain commented 1 day ago

After a couple of changes in TunnelBinding, the cloudflared deployment started to restarting endlessly.

2024-10-29T11:02:48Z    INFO    Selected protocol   {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "protocol": "http"}
2024-10-29T11:02:48Z    INFO    generated cloudflare config {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "hostname": "prerender.example.com", "target": "http://prerender.fastcup.svc:80"}
2024-10-29T11:02:48Z    INFO    Tunnel status is set    {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "status": {"hostnames":"prerender.example.com","services":[{"hostname":"prerender.example.com","target":"http://prerender.fastcup.svc:80"}]}}
2024-10-29T11:02:48Z    DEBUG   events  Configuring ConfigMap   {"type": "Normal", "object": {"kind":"TunnelBinding","namespace":"fastcup","name":"prerender","uid":"2605617a-7e99-4378-9e07-2195130e04e2","apiVersion":"networking.cfargotunnel.com/v1alpha1","resourceVersion":"6077115"}, "reason": "Configuring"}
2024-10-29T11:02:48Z    DEBUG   events  Applying ConfigMap to Deployment    {"type": "Normal", "object": {"kind":"TunnelBinding","namespace":"fastcup","name":"prerender","uid":"2605617a-7e99-4378-9e07-2195130e04e2","apiVersion":"networking.cfargotunnel.com/v1alpha1","resourceVersion":"6077115"}, "reason": "ApplyingConfig"}
2024-10-29T11:02:48Z    DEBUG   events  Applying ConfigMap to Deployment    {"type": "Normal", "object": {"kind":"Deployment","namespace":"cloudflare-operator-system","name":"fastcup","uid":"814b343c-65a3-4220-bb18-1c409761549c","apiVersion":"apps/v1","resourceVersion":"6077156"}, "reason": "ApplyingConfig"}
2024-10-29T11:02:48Z    INFO    Restarted deployment    {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0"}
2024-10-29T11:02:48Z    DEBUG   events  ConfigMap applied to Deployment {"type": "Normal", "object": {"kind":"TunnelBinding","namespace":"fastcup","name":"prerender","uid":"2605617a-7e99-4378-9e07-2195130e04e2","apiVersion":"networking.cfargotunnel.com/v1alpha1","resourceVersion":"6077115"}, "reason": "AppliedConfig"}
2024-10-29T11:02:48Z    DEBUG   events  ConfigMap applied to Deployment {"type": "Normal", "object": {"kind":"Deployment","namespace":"cloudflare-operator-system","name":"fastcup","uid":"814b343c-65a3-4220-bb18-1c409761549c","apiVersion":"apps/v1","resourceVersion":"6077176"}, "reason": "AppliedConfig"}
2024-10-29T11:02:48Z    DEBUG   events  Configured Cloudflare Tunnel    {"type": "Normal", "object": {"kind":"TunnelBinding","namespace":"fastcup","name":"prerender","uid":"2605617a-7e99-4378-9e07-2195130e04e2","apiVersion":"networking.cfargotunnel.com/v1alpha1","resourceVersion":"6077115"}, "reason": "Configured"}
2024-10-29T11:02:48Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "08a32c38-8008-4be1-8496-c7e47f6f629b"}
2024-10-29T11:02:48Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "08a32c38-8008-4be1-8496-c7e47f6f629b"}
2024-10-29T11:02:48Z    DEBUG   events  TunnelBinding Finalizer and Labels added    {"type": "Normal", "object": {"kind":"TunnelBinding","namespace":"fastcup","name":"prerender","uid":"2605617a-7e99-4378-9e07-2195130e04e2","apiVersion":"networking.cfargotunnel.com/v1alpha1","resourceVersion":"6077115"}, "reason": "MetaSet"}
2024-10-29T11:02:48Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "08a32c38-8008-4be1-8496-c7e47f6f629b", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:48Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "cb1f0236-13f2-481c-a39d-96e1f0f9806e"}
2024-10-29T11:02:48Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "cb1f0236-13f2-481c-a39d-96e1f0f9806e"}
2024-10-29T11:02:48Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "cb1f0236-13f2-481c-a39d-96e1f0f9806e", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:48Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "6cf62f1c-ea6d-4003-a8e1-19acb561fb8f"}
2024-10-29T11:02:48Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "6cf62f1c-ea6d-4003-a8e1-19acb561fb8f"}
2024-10-29T11:02:48Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "6cf62f1c-ea6d-4003-a8e1-19acb561fb8f", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:48Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "c80cf0d7-0403-4380-979f-b846a36d48ae"}
2024-10-29T11:02:48Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "c80cf0d7-0403-4380-979f-b846a36d48ae"}
2024-10-29T11:02:48Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "c80cf0d7-0403-4380-979f-b846a36d48ae", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:48Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "7b1bac63-e396-479f-91f0-a89fe93e27e8"}
2024-10-29T11:02:48Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "7b1bac63-e396-479f-91f0-a89fe93e27e8"}
2024-10-29T11:02:48Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "7b1bac63-e396-479f-91f0-a89fe93e27e8", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:50Z    INFO    Updating existing record    {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "fqdn": "prerender.example.com", "dnsId": "redacted"}
2024-10-29T11:02:50Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "97105764-b203-4b10-a27e-17ba540c4c7c"}
2024-10-29T11:02:50Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "97105764-b203-4b10-a27e-17ba540c4c7c"}
2024-10-29T11:02:50Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "97105764-b203-4b10-a27e-17ba540c4c7c", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:50Z    INFO    In validation   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "1263b978-ca6a-43fb-b7a1-ed4b93896418"}
2024-10-29T11:02:50Z    INFO    Validation successful   {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "1263b978-ca6a-43fb-b7a1-ed4b93896418"}
2024-10-29T11:02:50Z    INFO    Tunnel status is set    {"controller": "clustertunnel", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "ClusterTunnel", "ClusterTunnel": {"name":"fastcup"}, "namespace": "", "name": "fastcup", "reconcileID": "1263b978-ca6a-43fb-b7a1-ed4b93896418", "status": {"tunnelId":"redacted","tunnelName":"fc-hetzner-k8s","accountId":"redacted","zoneId":"redacted"}}
2024-10-29T11:02:50Z    INFO    DNS record updated successfully {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "fqdn": "prerender.example.com"}
2024-10-29T11:02:50Z    INFO    Updating existing TXT record    {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "fqdn": "prerender.example.com", "dnsId": "redacted", "txtId": "redacted"}
2024-10-29T11:02:51Z    INFO    DNS record updated successfully {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0", "fqdn": "prerender.example.com"}
2024-10-29T11:02:51Z    INFO    Inserted/Updated DNS/TXT entry  {"controller": "tunnelbinding", "controllerGroup": "networking.cfargotunnel.com", "controllerKind": "TunnelBinding", "TunnelBinding": {"name":"prerender","namespace":"fastcup"}, "namespace": "fastcup", "name": "prerender", "reconcileID": "ece5d62e-0a70-42c5-8f9e-b4c5805eb9e0"}
2024-10-29T11:02:51Z    DEBUG   events  Inserted/Updated DNS/TXT entry  {"type": "Normal", "object": {"kind":"TunnelBinding","namespace":"fastcup","name":"prerender","uid":"2605617a-7e99-4378-9e07-2195130e04e2","apiVersion":"networking.cfargotunnel.com/v1alpha1","resourceVersion":"6077115"}, "reason": "CreatedDns"}
maxpain commented 1 day ago

The problem only exists if using Flux CD. I have this TunnelBinding config:

apiVersion: networking.cfargotunnel.com/v1alpha1
kind: TunnelBinding
metadata:
  name: prerender
subjects:
  - name: prerender
tunnelRef:
  kind: ClusterTunnel
  name: fastcup

And cloudflare-operator changes subjects, adding spec:

subjects:
  - kind: Service
    name: prerender
    spec:
      noTlsVerify: false
      proxyAddress: 127.0.0.1
      proxyPort: 0
      proxyType: ''

After this, Flux CD reconciles this Custom Resource again.

maxpain commented 1 day ago

Why does cloudflare-operator change this CR at all? Is it necessary?