search

kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.68k stars 2.56k forks source link

hetzner: add support to select loadbalancer ip/hostname #1816

Closed tobiaskohlbau closed 3 years ago

tobiaskohlbau commented 4 years ago

Hetzner introduced load balancers in their hcloud some months ago. A loadbalancer can have a public and a private ip assigned. Both of these IP addresses get assigned to the service resource of k8s. If both of them are used the current behavior selects the internal IP as it is always the first. There is no possibility (as far as I know) to change the ip/hostname which external-dns uses to populate dns entries.

Reference to implementation which hard codes the first IP/Endpoint in a service resource: https://github.com/kubernetes-sigs/external-dns/blob/755edb400fad7c32db1f754cfd0f22914f4e918f/provider/hetzner/hetzner.go#L228-L233

What would you like to be added:

An annotation for e.g. external-dns.alpha.kubernetes.io/hcloud-loadbalancer-ingress-index to select the index which ip/hostname should be used when populating the DNS record.

Why is this needed:

To enable selection of either public or private IP address for DNS records.

Additional context For anyone curios to see the requested feature in a draft have a look at https://github.com/tobiaskohlbau/external-dns/commit/bc101209ef2c215077436a3f339fd58d1d0dbc55.

If this suggestion gets positive feedback I'm eager to align my CL with the project and bring it upstream.

tobiaskohlbau commented 4 years ago

Pinging @21h as he maintains the hetzner provider.

21h commented 4 years ago

Pinging @21h as he maintains the hetzner provider.

I'll think about it when I get some free time, I will need this feature soon too.

plicon commented 3 years ago

@tobiaskohlbau I'm running into the exact same issue, did you find a way around it?

tobiaskohlbau commented 3 years ago

Yes I gathered some options. First of all it's not hcloud which reorders the IPs. Instead external-dns itself reorders the items before forwarding them to each provider interface.

See for e.g.: https://github.com/kubernetes-sigs/external-dns/blob/65087c4e0246c6f621e54c34e9f56bffd2b763b9/source/virtualservice.go#L197-L199

The best possible way to get this working without running a modified external-dns is disabling the private network. https://github.com/hetznercloud/hcloud-cloud-controller-manager/blob/8d4e4669f654aa7660ab0f811c573ce6103f1617/internal/annotation/load_balancer.go#L27-L34

It's still possible to route the traffic from loadbalancer via private subnet to the nodes. Only internal nodes aren't able to leverage the LB. But this should most likely not be an issue as internal services could use kubernetes loadbalancing features. If it's important to have an internal LB for legacy apps, I would suggest to use two different LBs for now.

plicon commented 3 years ago

@tobiaskohlbau thanks for your reply. I also opened an issue on the hcoud-cloud-controller repo and was just directed there to the same option which i totally overlooked: adding the annotation "load-balancer.hetzner.cloud/disable-private-ingress=true" to the LB service did the trick for me.

tobiaskohlbau commented 3 years ago

Just for reference I just found that external-dns already has support for private and public ip selection: https://github.com/kubernetes-sigs/external-dns/blob/master/docs/faq.md#how-do-i-specify-that-i-want-the-dns-record-to-point-to-either-the-nodes-public-or-private-ip-when-it-has-both

I will try to dig into the code to find which parts are affected by that annotation and if it's possible to use that annotation to solve this issue.

tobiaskohlbau commented 3 years ago

Unfortunately that annotation couldn't be used as it's intended for services of type NodePort where an ip has the associated information of either being public or private. Whereas in LoadBalancer type the status.ip has no information if the IP is public or private. There I see two possible options:

  1. The option proposed in my fork where the ip is selected via index.
  2. An heuristics which parses the ip and identifies as either being public or private.

I would prefer 1 over 2 as in the second option the definition of private and public is being forced onto the user instead of being defined by the user.

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

k8s-ci-robot commented 3 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/1816#issuecomment-818315353): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.