kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.75k stars 2.58k forks source link

Not working configuration Gateway API #4768

Open didlawowo opened 2 months ago

didlawowo commented 2 months ago

-->

What happened: try to configure external dns with gateway api to provide dns entry

What you expected to happen: nslookup working

How to reproduce it (as minimally and precisely as possible): create a gateway / httproute with annotation

Anything else we need to know?: don't see anything in logs

Environment: k3s

my config

kind: Namespace
apiVersion: v1
metadata:
  name: whoami
  labels:
    app: whoami
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
        resources:
          requests:
            cpu: 100m
            memory: 20Mi
          limits:
            cpu: 100m
            memory: 20Mi
---
# Service to reach the application on the cluster
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: whoami
    # annotations:
    #   external-dns.alpha.kubernetes.io/hostname: whoami.dc-tech.work
spec:
  type: ClusterIP
  ports:
  - port: 80
    name: whoami
  selector:
    app: whoami
---

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  namespace: whoami
  name: my-whoami-gateway
  labels:
    app: whoami
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-cloudflare
spec:
  gatewayClassName: traefik-gateway
  listeners:
  - hostname: whoami.dc-tech.work
    name: hello-https
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: whoami-tls

---
# HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
  annotations:
    external-dns.alpha.kubernetes.io/target: whoami.dc-tech.work
spec:
  parentRefs:
  - name: my-whoami-gateway
    namespace: whoami
  hostnames:
  - whoami.dc-tech.work
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: whoami
      namespace: whoami
      port: 80

Pod info

Name:             external-dns-cloudflare-77c85cb8b5-l2cbd
Namespace:        kube-infra
Priority:         0
Service Account:  external-dns-cloudflare
Node:             rtx/192.168.1.29
Start Time:       Sun, 22 Sep 2024 23:07:37 +0200
Labels:           app.kubernetes.io/instance=external-dns-cloudflare
                  app.kubernetes.io/name=external-dns-cloudflare
                  pod-template-hash=77c85cb8b5
Annotations:      <none>
Status:           Running
SeccompProfile:   RuntimeDefault
IP:               10.0.0.63
IPs:
  IP:           10.0.0.63
Controlled By:  ReplicaSet/external-dns-cloudflare-77c85cb8b5
Containers:
  external-dns:
    Container ID:  containerd://becd130ce3f1128f09a8da0b2cbfac30d38f7b77c78b8efe7ef5dc26d941a16b
    Image:         registry.k8s.io/external-dns/external-dns:v0.15.0
    Image ID:      registry.k8s.io/external-dns/external-dns@sha256:338dd8c526e1337a12db0c8ee81a59ce5815ea51b73756affcd1e5dca530105c
    Port:          7979/TCP
    Host Port:     0/TCP
    Args:
      --log-level=debug
      --log-format=text
      --interval=1m
      --source=gateway-httproute
      --source=gateway-tlsroute
      --source=gateway-tcproute
      --source=ingress
      --source=service
      --policy=sync
      --registry=txt
      --txt-prefix=external-dns
      --domain-filter=dc-tech.work
      --provider=cloudflare
    State:          Running
      Started:      Sun, 22 Sep 2024 23:07:38 +0200
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:http/healthz delay=10s timeout=5s period=10s #success=1 #failure=2
    Readiness:      http-get http://:http/healthz delay=5s timeout=5s period=10s #success=1 #failure=6
    Environment:
      CF_API_TOKEN:  <set to the key 'cloudflare-api-token' in secret 'cloudflare-api-token'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-n7n92 (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-n7n92:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  5m38s  default-scheduler  Successfully assigned kube-infra/external-dns-cloudflare-77c85cb8b5-l2cbd to rtx
  Normal  Pulled     5m38s  kubelet            Container image "registry.k8s.io/external-dns/external-dns:v0.15.0" already present on machine
  Normal  Created    5m38s  kubelet            Created container external-dns
  Normal  Started    5m38s  kubelet            Started container external-dns
kundan2707 commented 1 month ago

/assign

didlawowo commented 1 month ago

i have also tested an another way and i get this error

leTLSInsecureSkipVerify:false PluralCluster: PluralProvider: WebhookProviderURL:http://localhost:8888 WebhookProviderReadTimeout:5s WebhookProviderWriteTimeout:10s Webhook │ │ Server:false TraefikDisableLegacy:false TraefikDisableNew:false}" │ │ time="2024-10-03T14:51:40Z" level=debug msg="apiServerURL: " │ │ time="2024-10-03T14:51:40Z" level=debug msg="kubeConfig: " │ │ time="2024-10-03T14:51:40Z" level=info msg="Using inCluster-config based on serviceaccount-token" │ │ time="2024-10-03T14:51:40Z" level=info msg="Created GatewayAPI client https://10.43.0.1:443" │ │ time="2024-10-03T14:51:40Z" level=info msg="Instantiating new Kubernetes client" │ │ time="2024-10-03T14:51:40Z" level=debug msg="apiServerURL: " │ │ time="2024-10-03T14:51:40Z" level=debug msg="kubeConfig: " │ │ time="2024-10-03T14:51:40Z" level=info msg="Using inCluster-config based on serviceaccount-token" │ │ time="2024-10-03T14:51:40Z" level=info msg="Created Kubernetes client https://10.43.0.1:443" │ │ time="2024-10-03T14:52:40Z" level=fatal msg="failed to sync *v1.HTTPRoute: context deadline exceeded" │ │ Stream closed EOF for whoami/external-dns-gateway-695c8d9dbf-pxv48 (external-dns-gateway)

i'm using gateway standard spec.

robsumoza commented 1 month ago

You need to add this configuration to your traefik deployment:

https://doc.traefik.io/traefik/providers/kubernetes-gateway/#statusaddress

As extra args it would be:

args:
- --providers.kubernetesGateway.statusAddress.service.namespace=<traefik_namespace>
- --providers.kubernetesGateway.statusAddress.service.name=<traefik_service_name>
adelmoradian commented 2 weeks ago

I also have the same issue using Istio with httproutes.gateway.networking.k8s.io. Nothing happens on external-dns when I create the http route with host name in annotation and spec but eventually external-dns crashes with msg="failed to sync *v1beta1.Gateway: context deadline exceeded". I've deployed external-dns with --source=gateway-httproute flag.

josemrs commented 2 days ago

I'm getting this annoying issue after I added

to my externalDNS config and now pods are failing to start and end up in CrashBackLoopOff, however, just restarting the deployment seems to do enough to et them working again. I don't understand.


{"level":"info","msg":"Using inCluster-config based on serviceaccount-token","time":"2024-11-25T16:26:54Z"}
{"level":"info","msg":"Created Kubernetes client https://172.20.0.1:443","time":"2024-11-25T16:26:54Z"}
{"level":"info","msg":"Using inCluster-config based on serviceaccount-token","time":"2024-11-25T16:26:55Z"}
{"level":"info","msg":"Created GatewayAPI client https://172.20.0.1:443","time":"2024-11-25T16:26:55Z"}
{"level":"fatal","msg":"failed to sync *v1alpha3.Gateway: context deadline exceeded","time":"2024-11-25T16:27:55Z"}```
josemrs commented 2 days ago

Actually, now I've seen better, is that pod starts and, at some point get into that timeout and get stuck. What is causing this error? IDK, it looks like and slow endpoint but ExternalDNS should not get stuck on this error anywat