[Feature] Using a defined subnet as the service CIDR

siegenthalerroger commented 2 years ago

Is your feature request related to a problem? Please describe. I've read the documentation (and issues) regarding the service CIDR and keep coming to the same conclusion. I don't see a way to prevent the possible dual-usage of the service-CIDR range. Of course there's an error warning if I define a service-CIDR that overlaps an existing subnet (https://github.com/Azure/AKS/issues/1117) and there's guidance that the service-CIDR isn't routed, etc. There's even a nice alert in the Azure Portal that the service-CIDR shouldn't be within the VNET range at all. (https://github.com/MicrosoftDocs/azure-docs/issues/12929)

HOWEVER, with the valid guidance being that the service-CIDR shouldn't be included in the VNET that the cluster is is on there is a big issue of potential collisions. Yes, no one in the same VNET can deploy a service that could potentially be masked by my service-CIDR, but as soon as someone peers another VNET you can run in to this problem essentially immediately.

Describe the solution you'd like I'd like to be able to have a way to protect my service-CIDR from being used in another, routed, competence in ANY of the peered VNETs. Taking a look at GKE we can see that it is solved there by using a subnet on the VNET just for the services, regardless of its routed-ness.

Describe alternatives you've considered Using rules to ensure that services that are available globally must be in the 10.x range, internal stuff 172.x (e.g. services) and local 192.168.x (e.g. docker bridge) is an idea, I'm unsure how to ensure that these rules are followed enterprise wide.

Additional context I've never seen this discussion in the AKS space at all, it seems that the network range discussion always ends at the VNET level and skips any peering concerns. Maybe I'm misunderstanding something foundational?

siegenthalerroger commented 2 years ago

I raised a support request with Azure support about this and it is as I understood it that you can run in to IP conflicts (however unlikely) with the current setup. As such I'd still like to have this feature to prevent that from happening at all.