Support multiple namespaces for Service sources

[bukovjanmic]

We would like to have an ability to watch for resources (annotated Services in our case) only in selected namespaces.

We have a multi-tenant cluster, where each tenant has a set of namespaces identified by a custom label.

Each tenant has its own DNS zone. What we need to achieve that a tenant can only specifiy hostnames belonging to its own zone.

For example, lets have two tenants, each tenant has a DNS zone - tenant1.example.com, tenant2.example.com. We need for the tenant to be able to specify a hostname belonging only to its own zone, e.g. tenant1 can only annotate a service with myservice.tenant1.example.com. If the tenant1 uses annotation like myservice.tenant2.example.com, it should be ignored.

One way to achieve this is to have multiple ExternalDNS operators, each watching a set of namespaces identified by a label and handling a zone for each tenant.

However, it seems not to be possible with current implementation. ExternalDNS can only watch all namespaces, or we can do multiple instances but each can watch only a single namespace.

[johngmyers]

/help /good-first-issue

[k8s-ci-robot]

@johngmyers: This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/3565): >/help >/good-first-issue > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[ghost]

/assign if this issue is available for work on, I am interested to work on this.

[ghost]

Hi @johngmyers,

I'm relatively new to Kubernetes, and this is my inaugural endeavor into open-source contribution. Please correct me if I'm mistaken.

Upon examining the issue, it seems that to enable support for multiple namespaces within service sources, we'd need a kubeinformers function that can monitor multiple namespaces simultaneously (as referenced in source/service.go on line 79):

informerFactory := kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, 0, kubeinformers.WithNamespace(namespace))

However, as per my understanding from this issue: https://github.com/kubernetes/kubernetes/issues/74415, kubeinformer doesn't currently support this feature.

An alternative approach could involve employing a loop or similar construct. Yet, this might entail a substantial refactoring. Would you consider this effort worthwhile? I appreciate your guidance.

[Shin-John]

@bukovjanmic Is this issue still relevant? I'd like to pick it up and was wondering if I could get more information about this issue.

/assign

[bukovjanmic]

Still relevant for us, though we solved it by using cluster-wise operator and writing custom admission controller which checks this.

Not having to do this would allow us to have greater flexibility, if there was an operator for a set of namespaces (identified by label) we could have a different DNS resolver for each tenant.

[Shin-John]

So would the desired result be an operator published to a place like operatorhub.io or is there a certain area in the codebase that should be worked on?

Additionally, may I know of the cluster-wide operator that is currently being used to solve this issue?

On Thu, Jun 13, 2024 at 3:50 AM bukovjanmic @.***> wrote:

Still relevant for us, though we solved it by using cluster-wise operator and writing custom admission controller which checks this.

Not having to do this would allow us to have greater flexibility, if there was an operator for a set of namespaces (identified by label) we could have a different DNS resolver for each tenant.

— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/external-dns/issues/3565#issuecomment-2164887973, or unsubscribe https://github.com/notifications/unsubscribe-auth/APA5DKK5PYKT7JFUK55U3XTZHFFMRAVCNFSM6AAAAABJG53FVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRUHA4DOOJXGM . You are receiving this because you were assigned.Message ID: @.***>

[karlschriek]

We would actually like to be able to use external-dns on just a single namespace (or more specifically, roll out multiple external-dns instances, each scoped to a specific namespace). However, the behaviour we've observed so far prevents us from doing this due to one simple fact: we are using Istio virtual services (deployed within mynamespace1, mynamespace2 etc) that route from an "istio-gateway" Pod with a loadbalancer provisioned in the "istio-gateway" namespace.

So, the virtual service (where external-dns must grab the DNS address) sits in a different namespace than the gateway (where external-dns must grab the IP address that the the DNS must route to). If we deploy it with the flag "--namespace=mynamespace1" then it is not able to resolve the gateway IP address.

One solution to this would be to be able to set the multiple namespace flags, such as "--namespace=mynamespace1 --namespace=istio-gateway"

[bukovjanmic]

I think OLM does not support multiple namespaces currently.

Also, it would probably require to specify multiple configurations (for multiple tenants), which would probably require to significantly refactor at least the configuration part.

The desired state here is therefore to run multiple cluster-wide external-dns operators (installed e.g. via Helm chart), each watching its own set of namespaces.

Currently there is an existing switch --namespace, which allows to specify only a single namespace.

The proposed solution would be to be able to specify multiple namespaces in --namespace switch, in one or more ways:

• separated by comma (in the --namespace switch)
• as regex (again the --namespace switch, or possibly --namespace-regex switch)
• as a reference to label (possible a new --namespace-label switch, like --namespace-label=domain.cz/tenant:mytenantname, which would be used as a filter.

The best option would be the ability to specify the label as a standard Kubernetes MatchExpression/MatchLabel YAML construct, but it would be possibly awkward to pass this YAML construct to a command line argument (possibly via some JSON notation).

Maybe the operator container could take this automatically from some annotations on its own pod, though,

I think the major issue is what is supported by Kubernetes informers that are currently used in external-dns operator. Last time I checked, they support only watching all or a single namespace.

The end result would be a clusterwide operator ignoring namespaces that do not fall into the definition.