AKS Nodepool VMSS cannot be Listed with List-All

[AndreasMWalter]

Situation:

• We currently are trying to give tag based firewall rules to Virtual Machine Scale Sets in a Checkpoint Cloudguard.
• Checkpoint uses the Azure API to query the tags on the resources.
• Tags for the AKS VMSS are set, but the Checkpoint cannot see the VMSS even though they are displayed in the Azure Portal as Ready

Analysis: Using the following API URI VMSS (for at least AKS but maybe even others) cannot be listed for several hours after deployment:

https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachineScaleSets?api-version=2021-07-01

The Azure Portal is first querying all scalesets using Azure Resource Graph

#Powershell code since this was used during testing
$Headers = @{
    'Content-Type' = 'application/json'
    'Authorization' = "Bearer $($AzAccessToken.Token)"
    'Host' = 'management.azure.com'
    'CommandName' = 'HubsExtension.OldBrowse-Microsoft.Compute/virtualMachineScaleSets'
}
$Body = @"
{
    "subscriptions": [
      "e080e486-f810-4464-90ff-d3e948b0b076" #or any other favorite guid
    ],
    "query": "where type in~ ('Microsoft.Compute/virtualMachineScaleSets')|project id,name,type,location,subscriptionId,resourceGroup,kind,tags|sort by (tolower(tostring(name))) asc",
    "options": {
      "`$top": 1000,
      "dataset": "Basic"
    }
  }
"@
Invoke-RestMethod -Method POST -Headers $Headers -Uri "https://management.azure.com/providers/Microsoft.ResourceGraph/resources?api-version=2018-09-01-preview" -Body $Body

And then querys all the VMSS individually using

https://management.azure.com/subscriptions/e080e486-f810-4464-90ff-d3e948b0b076/resourceGroups/mc_rg-name-_aks-name_westeurope/providers/Microsoft.Compute/virtualMachineScaleSets/aks-default-35064155-vmss?api-version=2021-04-01"

Assumption: Checkpoint uses the list function on the Azure API https://docs.microsoft.com/en-us/rest/api/compute/virtual-machine-scale-sets/list-all This returns no ressources for up to several hours after the scaleset was created

Question: Is this a Bug in the API or as intended and Checkpoint may be using an incorrect API call?

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Drewm3, @avirishuv.

Issue Details

---

Situation: - We currently are trying to give tag based firewall rules to Virtual Machine Scale Sets in a Checkpoint Cloudguard. - Checkpoint uses the Azure API to query the tags on the resources. - Tags for the AKS VMSS are set, but the Checkpoint cannot see the VMSS even though they are displayed in the Azure Portal as Ready Analysis: Using the following API URI VMSS (for at least AKS but maybe even others) cannot be listed for several hours after deployment: ``` https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachineScaleSets?api-version=2021-07-01 ``` The Azure Portal is first querying all scalesets using Azure Resource Graph ``` #Powershell code since this was used during testing $Headers = @{ 'Content-Type' = 'application/json' 'Authorization' = "Bearer $($AzAccessToken.Token)" 'Host' = 'management.azure.com' 'CommandName' = 'HubsExtension.OldBrowse-Microsoft.Compute/virtualMachineScaleSets' } $Body = @" { "subscriptions": [ "e080e486-f810-4464-90ff-d3e948b0b076" #or any other favorite guid ], "query": "where type in~ ('Microsoft.Compute/virtualMachineScaleSets')|project id,name,type,location,subscriptionId,resourceGroup,kind,tags|sort by (tolower(tostring(name))) asc", "options": { "`$top": 1000, "dataset": "Basic" } } "@ Invoke-RestMethod -Method POST -Headers $Headers -Uri "https://management.azure.com/providers/Microsoft.ResourceGraph/resources?api-version=2018-09-01-preview" -Body $Body ``` And then querys all the VMSS individually using ``` https://management.azure.com/subscriptions/e080e486-f810-4464-90ff-d3e948b0b076/resourceGroups/mc_rg-name-_aks-name_westeurope/providers/Microsoft.Compute/virtualMachineScaleSets/aks-default-35064155-vmss?api-version=2021-04-01" ``` Assumption: Checkpoint uses the list function on the Azure API https://docs.microsoft.com/en-us/rest/api/compute/virtual-machine-scale-sets/list-all This returns no ressources for up to several hours after the scaleset was created Question: Is this a Bug in the API or as intended and Checkpoint may be using an incorrect API call?

Author:	AndreasMWalter
Assignees:	ruowan
Labels:	`question`, `Compute - VMSS`, `Service Attention`, `customer-reported`, `needs-triage`
Milestone:	-

[amjads1]

@avirishuv Can you please look into this issue?

[avirishuv]

hi @AndreasMWalter are you still facing this issue?

[AndreasMWalter]

Ah sorry we opened a Microsoft case and also tried to recreate the issue half a year later, however it could no longer be reproduced. Interestingly according to the Case nothing was changed in the mean time in the API. But I am certain that I could reliably reproduce the issue in October on multiple subscriptions. So well no one knows and it might reoccur at some point.