search

kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.61k stars 2.55k forks source link

aws-target-hosted-zone annotation is buggy when having overlapping zones #3587

Closed interone-ms closed 6 months ago

interone-ms commented 1 year ago

What happened: When using the external-dns.alpha.kubernetes.io/aws-target-hosted-zone annotation in an environment with multiple overlapping zones, external-dns tries to create records in zones other than the one specified in the annotation.

What you expected to happen: external-dns should only attempt to create records in specified zones.

How to reproduce it (as minimally and precisely as possible):

  1. Create two Route53 private DNS zones: example.com (ID masked to ZONEA in logs) and foo.example.com (ID masked to ZONEB)
  2. Create an ingress with the host be.foo.example.com 
    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/scheme: internal
external-dns.alpha.kubernetes.io/aws-target-hosted-zone: "xxxxx" # ZONEB / foo.example.com
spec:
ingressClassName: alb
rules:
- host: be.foo.example.com
  http:
    paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: be
            port:
              number: 80
  3. Observe the logs, notice it did not create records in either zone: 
    time="2023-05-03T17:21:40Z" level=info msg="Applying provider record filter for domains: [example.com. .example.com. foo.example.com. .foo.example.com]"
time="2023-05-03T17:21:40Z" level=info msg="Desired change: CREATE cname-be.foo.example.com TXT [Id: /hostedzone/ZONEB]"
time="2023-05-03T17:21:40Z" level=info msg="Desired change: CREATE be.foo.example.com A [Id: /hostedzone/ZONEB]"
time="2023-05-03T17:21:40Z" level=info msg="Desired change: CREATE be.foo.example.com TXT [Id: /hostedzone/ZONEB]"
time="2023-05-03T17:21:41Z" level=error msg="Failure in zone foo.example.com. [Id: /hostedzone/ZONEB] when submitting change batch: InvalidChangeBatch: [Tried to create an alias that targets internal-k8s-xxx.eu-central-1.elb.amazonaws.com., type A in zone ZONEB, but the alias target name does not lie within the target zone, Tried to create an alias that targets internal-k8s-xxx.eu-central-1.elb.amazonaws.com., type A in zone ZONEB, but that target was not found]\n\tstatus code: 400, request id: 741d9e41-feb3-4bab-bc52-7b0f1bb353e1"
time="2023-05-03T17:21:42Z" level=info msg="Desired change: CREATE cname-be.foo.example.com TXT [Id: /hostedzone/ZONEA]"
time="2023-05-03T17:21:42Z" level=info msg="Desired change: CREATE be.foo.example.com A [Id: /hostedzone/ZONEA]"
time="2023-05-03T17:21:42Z" level=info msg="Desired change: CREATE be.foo.example.com TXT [Id: /hostedzone/ZONEA]"
time="2023-05-03T17:21:42Z" level=error msg="Failure in zone example.com. [Id: /hostedzone/ZONEA] when submitting change batch: InvalidChangeBatch: [RRSet with DNS name be.foo.example.com., type Acontains an alias target that contains a hosted zone ZONEB that is an invalid alias target., Tried to create an alias that targets internal-k8s-xxx.eu-central-1.elb.amazonaws.com., type A in zone ZONEB, but the alias target name does not lie within the target zone]\n\tstatus code: 400, request id: dafd3d0b-960f-4df5-afb5-6a21b860d4d0"
time="2023-05-03T17:21:43Z" level=error msg="failed to submit all changes for the following zones: [/hostedzone/ZONEB /hostedzone/ZONEA]"

Anything else we need to know?:

Environment:

johngmyers commented 1 year ago

That annotation is not documented to do what you are expecting it to do.

piotrb commented 1 year ago

Apologies for piggybacking on this issue .. there seems to be an old closed issue around this as well ..

https://github.com/kubernetes-sigs/external-dns/issues/3128

It feels like the OP is trying to basically solve the same issue with this annotation ..

is there any way to influence which zone the controller will chose to add records to in the event of overlapping zone names at all?

I'd really take any of the following:

johngmyers commented 1 year ago

According to the code, the AWS provider will apply changes to the public zone with the longest domain, if any, plus all matching private zones. This is not configurable other than by excluding zones outright.

m00lecule commented 1 year ago

Hi, Today I faced this issue. I have scenario when records destination hostedzone must be explicitly specified (private, public or both). I wanted to solve it using Ingress annotation but probably I will end up with three external-dns helm releases.

martin31821 commented 1 year ago

I've just ran into this issue as well, in our case we have a split-dns setup with private and public hosted zones for the same domain name, and I've tried to separate these with the annotation.

@piotrb: I'd prefer specifying the exact zone ID to use, much as if the entire deployment was scoped to this specific zone id. This might even be possible to implement in a provider-agnostic manner.

If such a solution is a viable way forward, i'd be willing to contribute a PR for it.

piotrb commented 1 year ago

@martin31821 given your use case, yep .. you definitely can only use the zone id .. my options were a bit more aligned with ease of use :) I wouldn't want expect my fellow developers to be specifying zone id's in specs unless the usecase absolutely required it (eg. yours)

interone-ms commented 1 year ago

It's the same case for us, not to mention that having the same record in multiple zones can be confusing and is, frankly, a pollution for the top-level domain's zone file.

k8s-triage-robot commented 8 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 6 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 6 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/3587#issuecomment-2024499479): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.