An additional resource group is created when deploying AKS

[OguzPastirmaci]

When deploying an AKS cluster, an additional resource group is created.

The resource group that I created and deployed AKS to:

az resource list -g oguzp-aks
Name       ResourceGroup    Location    Type                                        Status
---------  ---------------  ----------  ------------------------------------------  --------
oguzp-aks  oguzp-aks        westus2     Microsoft.ContainerService/managedClusters

The resource group that was created automatically:

az resource list -g MC_oguzp-aks_oguzp-aks_westus2
Name                                                                 ResourceGroup                   Location    Type                                          Status
-------------------------------------------------------------------  ------------------------------  ----------  --------------------------------------------  --------
agentpool1-availabilitySet-14710316                                  MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/availabilitySets
aks-agentpool1-14710316-0_OsDisk_1_fff6a42716dd4dc0a1032afc1cb67091  MC_OGUZP-AKS_OGUZP-AKS_WESTUS2  westus2     Microsoft.Compute/disks
aks-agentpool1-14710316-1_OsDisk_1_ff41571a0143470bbfc3b62653df5c2c  MC_OGUZP-AKS_OGUZP-AKS_WESTUS2  westus2     Microsoft.Compute/disks
aks-agentpool1-14710316-2_OsDisk_1_eec530dff34d4bdf80bbac8e74f5e07d  MC_OGUZP-AKS_OGUZP-AKS_WESTUS2  westus2     Microsoft.Compute/disks
aks-agentpool1-14710316-0                                            MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/virtualMachines
aks-agentpool1-14710316-0/cse0                                       MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/virtualMachines/extensions
aks-agentpool1-14710316-0/OmsAgentForLinux                           MC_OGUZP-AKS_OGUZP-AKS_WESTUS2  westus2     Microsoft.Compute/virtualMachines/extensions
aks-agentpool1-14710316-1                                            MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/virtualMachines
aks-agentpool1-14710316-1/cse1                                       MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/virtualMachines/extensions
aks-agentpool1-14710316-1/OmsAgentForLinux                           MC_OGUZP-AKS_OGUZP-AKS_WESTUS2  westus2     Microsoft.Compute/virtualMachines/extensions
aks-agentpool1-14710316-2                                            MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/virtualMachines
aks-agentpool1-14710316-2/cse2                                       MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Compute/virtualMachines/extensions
aks-agentpool1-14710316-2/OmsAgentForLinux                           MC_OGUZP-AKS_OGUZP-AKS_WESTUS2  westus2     Microsoft.Compute/virtualMachines/extensions
aks-agentpool1-14710316-nic-0                                        MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Network/networkInterfaces
aks-agentpool1-14710316-nic-1                                        MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Network/networkInterfaces
aks-agentpool1-14710316-nic-2                                        MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Network/networkInterfaces
aks-agentpool-14710316-nsg                                           MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Network/networkSecurityGroups
aks-agentpool-14710316-routetable                                    MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Network/routeTables
aks-vnet-14710316                                                    MC_oguzp-aks_oguzp-aks_westus2  westus2     Microsoft.Network/virtualNetworks

[anhowe]

@OguzPastirmaci this is by design. This second resource group is the "cluster resource group" and is used to represent and hold the lifecycle of resources underneath it. What is the impact you are seeing by having the second resource group?

[OguzPastirmaci]

I guess there isn't a significant impact but it's just the fact that the experience is different than ACS before. I created an ACS cluster and didn't have a second resource group so I wasn't expecting a second resource group that I don't have any control over the naming etc.

Would that mean that I won't be able to deploy an AKS cluster if I have access rights only to a resource group in a subscription and not the subscription itself?

[rspaulino]

@anhowe I agree with @OguzPastirmaci too, for testing and dev is probably Ok but on production on a large company you want to have control over the resources that you have maybe even tag some resources for billing or department tracking.

Others Azure manage services don't behave this way like manage disk, SQL server, etc so I wonder want change that Microsoft decide to take this route.

[SenthuranSivananthan]

The impact that I see with my customer are:

1. Resource Groups are created with billing tags for charge back. The new cluster RG doesn't contain this information.

2. Customer has a naming convention for the RGs. The new RGs don't conform with their standards.

3. Users don't have Owner permissions on the subscription, therefore the newly created RG is not visible to the user that created the cluster. This means, get-credential fails with read permissions. Error:

The client '[email address]' with object id '[guid]' does not have authorization to perform action 'Microsoft.ContainerService/managedClusters/read' over scope '/subscriptions/[guid]/resourceGroups/aks-test1/providers/Microsoft.ContainerService/managedClusters/app1'.

Ideally, we can specify the cluster resource group so that the permissions and tags can be pre-created.

[seanknox]

Resource Groups are created with billing tags for charge back. The new cluster RG doesn't contain this information.

Sorry if this is obtuse, but can you just add the tags?

Customer has a naming convention for the RGs. The new RGs don't conform with their standards.

Is there an operational impact with this?

Users don't have Owner permissions on the subscription, therefore the newly created RG is not visible to the user that created the cluster.

Haven't seen this before. The sub that created the managed cluster should be an owner of the node pool agent group (e.g. MC_cluster_name...).

cc @sgoings @slack in case I'm missing something.

[SenthuranSivananthan]

Yes, the tags can be added via script so it won't be too big of a workaround. Would be nice if there's a way to override it w/ another switch (i.e. --cluster-resource-group).

No operationally impact with the RG name. It just deviates from the norm that they've defined.

[anhowe]

@SenthuranSivananthan great idea on the custom creation, I'll submit a feature request for this.

[evillgenius75]

I think the bigger issue is around Service Principals. Many times the user deploying will not have the rights on a subscription or be able to create a SP that has Subscription level scope. Many times SPs are handed out by Central IT admin, or security teams in Large orgs and those are also scoped to just the RG needed, not the entire subscription. I think MSI is the answer to all of this, but this is something I have heard from the preview users of ACS RPv2 which also creates another RG for resources and it is not liked by those that have already set governance models on their subscriptions

[SenthuranSivananthan]

@evillgenius75, SPs also have a lifecycle and the secrets are regenerated based on security policies so we need to enable the ability to change SPs & secrets as well. We can take this one in another thread later.

[weinong]

Regarding SP's scope, you will not need to apply anything at all. The RP will give contributor role to just the RG we create. This change will roll out in a week or two.

[ElectricWarr]

It's been almost two weeks, can anyone confirm if/when this change will be applied?

[marrobi]

I think being able to specify a custom name would be a good solution as suggested by @anhowe . Most organizations I work with use naming conventions for resource groups, such as AppName-EnvironmentName-Infra etc.

[yeniklas]

I need to create a public ip within the same resource-group, is there a way to fetch the resource-group name that a certain cluster is using? And yes, it would be very nice to be able to specify the cluster resource-group explicitly.

[slack]

@smoerboegen we do not include the node resource group in the managedCluster API, but that would be a great addition. I'll add that feature to our backlog.

[rwaal]

I currently work at a client that has a strict naming convention for resource groups. So the ability to control the name of the cluster resource group is a must-have.

Additionally, it would nice to be able to specify to have all AKS cluster resources in one single resource group. So the AKS cluster, as well as all other cluster resource, such as the VMs and network resources.

[derekperkins]

I also don't understand why the resources are put into a separate group. If I'm concerned that the AKS managed objects are going to clutter my namespace, then I can choose an empty resource group. Is the worry that people are going to edit/delete resources that AKS is relying on, then blame AKS?

[Addisco]

I just had the issue that the name of the additional resource group got too long. I deployed the AKS cluster in a group which name is 35 chars long, the name of the cluster resource is 36 chars long - resulting in 86 chars (including the MC_ prefix and _<regionname> suffix) for the resource group name. As a result the deployment of the availability set fails with the following error:

The entity name 'resourceGroupName' is invalid according to its validation rule: ^[^_\W][\w-._]{0,79}(?<![-.])$.

Nevertheless, the MC_* resource group was created.

[dmiyamasu]

This enhancement request is preventing us from making the migration from ACS to AKS. Please take into consideration when prioritizing the backlog.

[mallikharjunrao]

Hi,

Facing the similar issue. Azure creating the additional resource group. And when i am trying to get the credentails getting the below exception on both the actual resourse group and the azure create resource group. Kindly do the needful.

$az acs kubernetes get-credentials --resource-group=devops --name=zoom-k8s
The Resource 'Microsoft.ContainerService/containerServices/zoom-k8s' under resource group 'devops' was not found.

$ az acs kubernetes get-credentials --resource-group=MC_devops_zoom-k8s_westeurope --name=zoom-k8s
The Resource 'Microsoft.ContainerService/containerServices/zoom-k8s' under resource group 'MC_devops_zoom-k8s_westeurope' was not found.

[slack]

@mallikharjunrao makes sure you use the aks sub command when pulling credentials for an AKS cluster. From the output, looks like you are runing az acs ...

[gugu91]

Is there any plan to work on this?

Also, can the second RG be safely inferred? Let's say that for example I create a cluster called my-aks in my-aks-rg in westeurope, meaning it will always be MC_my-aks-rg_my-aks_westeurope?

It would be handy a command to retrieve this from the CLI (if there is not already) as long as the enhancement is not in place.

[slack]

@gugu91 we do not yet have plans to remove or hide the second resource group, it is something we are investigating. However, for folks who do need to safely acquire the additional resource group, we have added a new property nodeResourceGroup to the 2018-03-31 API:

$ az resource show --api-version 2018-03-31 \
  --namespace Microsoft.ContainerService \
  --resource-type managedClusters \
  -g <RESOURCEGROUP> -n <RESOURCENAME> -o json \
| jq .properties.nodeResourceGroup
"MC_foo-aks-centralus_foo-aks-centralus_centralus"
$

[gugu91]

@slack Thanks for the prompt response. I am inferring though that this should be used as a last resort. What is the suggested way of retrieving info about the autogenerated resource group? Can I simply use kubectl in order to, for example, retrieve the public IP of a load balancer?

[leonardocastanodiaz]

I am facing the same issue with the RG and naming conventions, somebody has found a workaround? Many thanks

[peterwy01]

Same issue here, please add the possibility to set the name of the node ressource group when creating the aks cluster.

[abouquet]

@slack , Ressources Group are part of a RBAC strategy in my company and are strictly managed (naming convention, access...) I think you should be able at least to specify the node ressource group at AKS cluster creation as stated by @smoerboegen , not only beeing able to retrieve the generated name as you've suggested.

Hope it will be improved quickly in order to be able to use this new service widely :)

[NathanNZ]

Resource Groups never seem to be fully supported in Azure. It seems very common for new Azure products to just throw resources in randomly generated resource groups, and then not allow you to move them out. "Use tagging instead, just throw in some custom scripts, use GreaseMonkey to rename it in your browser " really doesn't do anybody any favours.

I feel like there needs to be a little bit more empathy for people within businesses that like to have nice, tidy, auditable and easy to navigate subscriptions. They should not have to work around artificial limitations or be forced to work a specific way due to the arbitrary and inconsistent technical restrictions that do not give a consistent experience from product to product. Especially since resource groups were supposed to empower those using Azure to do it a way that made sense for them!

"You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization"

Now that AKS is GA, it would be great for us to get some more information about when this fits in the roadmap so we are one step closer to something that all mid to large businesses will happily move from ACS to.

[maadtechie]

I've added AKS cluster in existing resource group and existing vnet, than I noticed that I have two NEW resource groups:

1. MC_ExistingResourceGroupName_k8s_westeurope.
2. DefaultResourceGroup-WEU. (this group is empty) After reading this thread I understand why first resource group was created, but why does k8s cluster need two new resource groups? Can I delete the second one, which is empty??

[rdtechie]

I agree with @NathanNZ . Some customers of mine are looking to AKS with interest, but the fact that they don't control the naming of the resources is holding them back. The answer to the question if this is a operational issue is no, not really. But, these customers use strict naming policies and their operational teams are used to use these names to search something in the portal/command line. It would be nice if we can influence the naming by use of a ARM template or whatsoever.

[SamirFarhat]

Hi,

Can we at least specify tags on the RG crated automatically, any update ?

[mmisztal1980]

I'd apreciate na update on this. We have a strict naming policy in our company and we'd like to have the option to create all of the resources in the resource group of our choosing.

[GSTZ]

Does the Azure SDK for .NET provide the way to get the auto-generated resource group name?If yes, how?

[bwalding]

This is problematic from the security perspective - I don't want to grant the engineering team the ability to create arbitrary resource groups - as they inevitably can't follow naming conventions / forget about resources running etc.

So I have them locked into pre-created and correctly named resource groups - right up until AKS goes and creates new resource groups that the user can't access and manage. It would be ok if the user never needed to touch this new RG, but since they need access to those RGs to debug their solutions, they need to raise requests for us to unlock new RGs that AKS is creating.

[andreujuanc]

Being able to set a custom name for both RGs it's a must.

[piotr-napadlek]

Yep, when I want to use preexisting public IP address to assign it to a load balancer inside aks, I need to create it in the aks node resource group, which brings much more complexity to my deployment process. Would appreciate some control over resource group being created.

[atrauzzi]

Just created an AKS cluster now to try something out and saw all the resource group spam.

This needs to be fixed right away. Azure needs a strict policy against creating resource groups from wizards, it's just never okay to do this.

(unless that wizard is the create resource group wizard 😉 )

[andreujuanc]

Or, do the opposite, hide everything from us (the way it should actually be), but still give us the option of connecting to subnets, etc.

[wy193777]

Wasting the whole day on how to bind public ip to an AKS created RG. And find this thread.

[turowicz]

The MC_ prefix is really screwing with my OCD. I need all my resource groups be named in a specific way: {prefix}-{location}-{environment}-{project} lowercase.

[marrobi]

I vary rarely look inside the MC_ resource group now, and work with AKS on a daily basis. How about if it was hidden by default using RBAC maybe, but could be revealed for troubleshooting process? Or is the RG name used by other processes within your subscription? Thinking aloud here.

[atrauzzi]

I've always been a fan of this one: https://feedback.azure.com/forums/281804-azure-resource-manager/suggestions/8618338-nested-resource-groups

Allow nesting and rename of resource groups. Would probably solve zillions of customer gripes for Azure.

[jeliasson]

We're almost one year in this issue. Any solution?

[billtr]

Totally agreed that this is a big problem in many ways including security, standard, access and control. In an enterprise environment where you have a fixed set of subscriptions for people to use, typical users only have resource-group level contributor permission. From the customer perspective, we need the ability to have control over the creation of this second resource group manually or automatically. For AKS, it needs the ability to allow people to create new or use existing resource group that users create at deployment time, just like any other Azure services.

[mkosieradzki]

IMO it looks like a workaround for lack of VMSS support. I have a hunch that MC resource groups will go away as soon as they switch to VMSS in AKS. MC group contains all the resources that are usually managed by the VMSS.

But I might be wrong...

[NathanNZ]

https://feedback.azure.com/forums/914020-azure-kubernetes-service-aks/suggestions/35496043-please-create-all-the-kubernetes-related-resources

Looks like there is an active uservoice issue relating to this - worth taking the time and voting. Only would take a few people to get it to the front page.

[skyflyer]

Would it be OK to manually move resources from the auto-generated resource group to my chosen resource group? Is it even possible?

e.g., to move ContainerInsights and LogAnalytics workspace from MC_... group to my group? Would that even work, or would I need to reconfigure the AKS integration somehow to point it to the new resource IDs? (or are resource IDs preserved when moving them from one resource group to another)?

[MaurGi]

Can we at least provide our own name for the resource group that is being created?

[MagicMarty]

Would it be OK to manually move resources from the auto-generated resource group to my chosen resource group? Is it even possible?

e.g., to move ContainerInsights and LogAnalytics workspace from MC_... group to my group? Would that even work, or would I need to reconfigure the AKS integration somehow to point it to the new resource IDs? (or are resource IDs preserved when moving them from one resource group to another)?

I have tried this scenario and you are able to move resources. Then AKS cluster status on Azure shows as Failed. When logged into the cluster all nodes looks healthy anyway, but still cluster status stays as Failed. Antoher issue, if you scale, new resources will get created to the same MC_ resource group :)

And officialy by MS docs AKS services cannot be moved: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-move-resources#services-that-cannot-be-moved

[pascalgn]

Related issue: AKS cannot be used when certain resource group policies are in place: https://github.com/Azure/azure-cli/issues/7573

[pawelpabich]

A flag that allows us to put all resources in one RG would help quite a bit.