[BUG] upgrading to azure cni overlay triggers an linked scopes error

[diegopasten]

Describe the bug Not sure if bug, or just missing documentation.

I am working in an environment that is designed with Cloud Adoption Framework (CAF) as reference. When trying to to upgrade from Azure CNI to Azure CNI in overlay mode, I get an error with linked scopes to a log analytics workspace that is set up with policies according to the CAF-recommendations. In short I am on a team that does not own the log analytics workspace that is connected to the various clusters that this team manages. This Azure policy is global for the company.

To Reproduce Steps to reproduce the behavior:

1. az aks update --name $clusterName \
   --resource-group $resourceGroup \
   --network-plugin-mode overlay \
   --pod-cidr 192.168.0.0/16

2. (LinkedAuthorizationFailed) The client 'user@company.com' with object id '$USER_OBJECT_ID' has permission to perform action 'Microsoft.ContainerService/managedClusters/write' on scope '/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$resourceGroup/providers/Microsoft.ContainerService/managedClusters/$clusterName'; however, it does not have permission to perform action(s) 'Microsoft.OperationalInsights/workspaces/sharedkeys/read' on the linked scope(s) '/subscriptions/$LOGAW_Subscription_ID/resourcegroups/$LOGAW_ResourceGroup/providers/Microsoft.OperationalInsights/workspaces/$CentralLogAW' (respectively) or the linked scope(s) are invalid.
   Code: LinkedAuthorizationFailed

Expected behavior I expect it to take down all nodes and change the network stack without having to read or modify the connected log analytics workspace. If this is intended and/or impossible to avoid then I think it should also be mentioned in the docs because we are following a Microsoft CAF-pattern.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• Azure CLI Version [e.g. 2.54.0]
• Kubernetes version [e.g. 1.27.7]
• CLI Extension version [e.g. 1.7.5] if applicable
• Browser [e.g. chrome, safari] is applicable

Additional context I have been able to apply the command If I first destroy the connection to the logaw.

[VincentVerweij]

@diegopasten it has been a few months since you reported this. How were you and your team able to resolve this?

I had a similar issue when running the following:

az aks update --resource-group rg-*** --name aks-*** --enable-oidc-issuer --enable-workload-identity --no-wait

Came across this issue and issue 22232 on the azure-cli repo.

The error is very similar to yours, as it also states that I need read access on the Log Analytics Workspace:

ERROR: (LinkedAuthorizationFailed) The client 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' with object id 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' has permission to perform action 'Microsoft.ContainerService/managedClusters/write' on scope '/subscriptions/***/resourceGroups/rg-***/providers/Microsoft.ContainerService/managedClusters/aks-***'; however, it does not have permission to perform action(s) 'Microsoft.OperationalInsights/workspaces/sharedkeys/read' on the linked scope(s) '/subscriptions/***/resourceGroups/rg-***/providers/Microsoft.OperationalInsights/workspaces/log-***' (respectively) or the linked scope(s) are invalid.

In the other issue, someone suggests adding Log Analytics Reader role to the user.

[VincentVerweij]

Getting back on my comment:

In the other issue, someone suggests adding Log Analytics Reader role to the user.

This does not appear to be working anymore, I have resolved this by adding the Log Analytics Contributor role.

[diegopasten]

Hi @VincentVerweij !

We ended up creating a custom role with the IAM-team which had exactly the necessary permissions.

This was needed as this issue was also starting to prevent us from upgrading Kubernetes versions on nodes and control plane. The downside is that every service principal that manipulates the cluster needs to have this role assigned. So we have started migration to LAWs that are controlled by our team.

[FrancoisPoinsot]

I faced the same issue. I was running an even simpler udpate command

az aks update --name $clusterName --resource-group $resourceGroup

No argument, just to trigger reconciliation. Same error. Tried after granting Log Analytics Reader: didn't change the error. Granting Log Analytics Contributor "solved" the problem.

Except that's a bit more permissions that I would have liked. And according to the error message, a simple Reader should have been enough.