search

kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.67k stars 2.56k forks source link

Disable external-dns for specific ingresses #819

Closed StianOvrevage closed 3 years ago

StianOvrevage commented 5 years ago

We have external-dns set up with --source=ingress.

We have multiple clusters running variations of the same service. We then point a permanent DNS alias (CNAME) to the cluster that is operational.

For this to work we need to have ingresses across clusters that have the same hostname.

We want to have control over these CNAME records. However, external-dns keeps overwriting and creating race-conditions messing everything up.

Ideally there should be an ingress annotation allow us to disable external-dns for that ingress but I have not been able to find any such functionality.

Next I have tried adding --txt-prefix=lock. to allow TXT and CNAME to co-exist. I look at the lock.alias TXT record, change the owner-ref to something else and delete the A record to free it up so we can create the needed CNAME. A second later the TXT record is changed back and the A record is back preventing us from creating a CNAME to the chosen cluster.

wstewartii commented 5 years ago

I've been trying to figure out how to do this as well. I use the cert-manager controller to create letsencypt certificates and would like to use a single wildcard dns record and certificate to avoid rate limits when creating and deleting ingress resources.

ali-essam commented 5 years ago

As a hack, you can set the hostname to empty string on the ingress

external-dns.alpha.kubernetes.io/hostname: ""

prachidamle commented 5 years ago

Does this annotation "external-dns.alpha.kubernetes.io/hostname" apply for ingresses as well? or is it just for services of type=LB

ali-essam commented 5 years ago

@prachidamle According to personal testing and a quick look at the code, yes, it should support ingress

rverma-nikiai commented 5 years ago

with aws-alb-ingress-controller, setting external-dns.alpha.kubernetes.io/hostname: "" doesn't work. Even removing the annotation doesn't work.

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

rverma-nikiai commented 5 years ago

/remove-lifecycle stale

linki commented 5 years ago

@StianOvrevage I can give you a couple of ideas but I didn't understand your setup fully.

For CNAME to work with the TXT registry you have to use a non-empty --txt-prefix on all your ExternalDNS instances (it should be the same on all in your case)

When you have multiple ExternalDNS instances messing around with the same hostnames and zones you should give each instance a different non-empty value for their --txt-owner-id. This allows each instance to "own" a record so that once it's "claimed" it won't be change by other ExternalDNS instances even if they also know about that DNS name.

If you then want to change the "ownership" of a DNS record from one cluster to another you could manually update the owner value in the TXT record and the other ExternalDNS instance will adopt that record (and probably change the corresponding CNAME to its cluster's Ingress entrypoint).

If you merely want to instruct ExternalDNS to ignore certain Ingresses you could start them with the --annotation-filter flag which expects an annotation selector and matches all Ingresses of a cluster against it.

For example, you could start ExternalDNS with --annotation-filter=ingress.class=nginx to have it only process Ingresses that are backed by the nginx ingress controller. It supports the full label selector syntax, so you can also use != etc.

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 4 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 4 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 4 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/819#issuecomment-568291423): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
2tim commented 4 years ago

For example, you could start ExternalDNS with --annotation-filter=ingress.class=nginx to have it only process Ingresses that are backed by the nginx ingress controller. It supports the full label selector syntax, so you can also use != etc.

@linki How does the != syntax need to look? I've tried it a few ways and not been able to get it to work. I get the following error: "\"!=\" is not a valid label selector operator" using: --annotation-filter=kubernetes.io/ingress.class!=internal-ingress

runningman84 commented 4 years ago

/reopen

k8s-ci-robot commented 4 years ago

@runningman84: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/819#issuecomment-591629643): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
Nuru commented 4 years ago

For example, you could start ExternalDNS with --annotation-filter=ingress.class=nginx to have it only process Ingresses that are backed by the nginx ingress controller. It supports the full label selector syntax, so you can also use != etc.

@linki How does the != syntax need to look? I've tried it a few ways and not been able to get it to work. I get the following error: "\"!=\" is not a valid label selector operator" using: --annotation-filter=kubernetes.io/ingress.class!=internal-ingress

@linki the != syntax does not work

@2tim the syntax is described in this comment by @Evesy 

"--annotation-filter=kubernetes.io/ingress.class notin (internal-ingress)"
2tim commented 4 years ago

@Evesy thanks, I think I found that shortly after posting this buried somewhere. I’m using that today.

IronCore864 commented 3 years ago

I request a feature or a bug fix.

Setting external-dns.alpha.kubernetes.io/hostname: "" for AWS load balancer controller doesn't work.

I think ignoring one or two ingress objects is a valid requirement.

For example, you are creating an ALB which sends traffic to the website services, but you want to put your apex domain on a CloudFront which uses the ALB as an origin instead of putting the apex domain on the ALB itself.

Of course, there can be some manual hacks, for example, setting the policy to create-only instead of upcert-only or sync, then after the record is created you can change it to the CloudFront distribution. But you probably want to upcert or sync for all the other ingresses, so this won't work.

So, please, implement this. Some annotation like external-dns.alpha.kubernetes.io/create-dns-record: false which defaults to true would be nice.

IronCore864 commented 3 years ago

/reopen

k8s-ci-robot commented 3 years ago

@IronCore864: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/819#issuecomment-756027592): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
seanmalloy commented 3 years ago

@IronCore864 this issue is now reopened.

/reopen

k8s-ci-robot commented 3 years ago

@seanmalloy: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/819#issuecomment-758867567): >@IronCore864 this issue is now reopened. > >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
seanmalloy commented 3 years ago

/remove-lifecycle rotten

seanmalloy commented 3 years ago

We will use #1910 instead. Sorry for the confusion.

/close

k8s-ci-robot commented 3 years ago

@seanmalloy: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/819#issuecomment-758868790): >We will use #1910 instead. Sorry for the confusion. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
gerbyzation commented 1 year ago

You can set external-dns.alpha.kubernetes.io/ingress-hostname-source: annotation-only to force it to generate DNS records only based on the annotations. If you then leave out the external-dns.alpha.kubernetes.io/hostname annotation it won't generate any records at all.

Docs: https://github.com/kubernetes-sigs/external-dns/blob/master/docs/faq.md#how-do-i-specify-a-dns-name-for-my-kubernetes-objects

bakayolo commented 1 year ago

This does not work with istio virtual services :( We could add a check here -> https://github.com/kubernetes-sigs/external-dns/blob/master/source/istio_virtualservice.go#L330 Similar to how it's done in the ingress file Do you know if there any WIP for this?

You can set external-dns.alpha.kubernetes.io/ingress-hostname-source: annotation-only to force it to generate DNS records only based on the annotations. If you then leave out the external-dns.alpha.kubernetes.io/hostname annotation it won't generate any records at all.

Docs: https://github.com/kubernetes-sigs/external-dns/blob/master/docs/faq.md#how-do-i-specify-a-dns-name-for-my-kubernetes-objects

Note: I added external-dns.alpha.kubernetes.io/controller: hacktonotmanagethesehostswithexternaldns as a hack to make it works.