Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.54k stars 2.76k forks source link

Unable to retrieve complete resources and SKU data using the Azure SDK for Python based on my subscription. #36690

Closed lokesh190496 closed 1 week ago

lokesh190496 commented 1 month ago

@shanselman @osake Encountering issues with the Azure SDK for Python while attempting to retrieve all resources and their associated SKU data for a specific subscription. The data returned is incomplete, and I'm seeking guidance on using a single SDK or REST API to reliably fetch this information.

kashifkhan commented 1 month ago

Hi @lokesh190496 , can you provide more information about the problem you are seeing. We would want to know python version, names of the SDKs being used, minimal repro code etc which will help us route you to the right folks.

github-actions[bot] commented 1 month ago

Hi @lokesh190496. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

lokesh190496 commented 1 month ago

Hi @lokesh190496. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

@kashifkhan import json from azure.identity import ClientSecretCredential from azure.mgmt.resource import ResourceManagementClient from azure.mgmt.resource.resources.models import GenericResourceExpanded

config = { 'subscription_id':'subscription_id', 'client_id':'client_id', 'tenant': 'tenant', 'secret': 'secret' }

class AzureResourceFetcher: def init(self):

Authenticate using ClientSecretCredential

    self.credentials = ClientSecretCredential(
        client_id=config['client_id'],
        client_secret=config['secret'],
        tenant_id=config['tenant']
    )
    self.resource_client = ResourceManagementClient(self.credentials, config['subscription_id'])

def fetch_resources_metadata(self):
    # Fetch all resources
    resources = self.resource_client.resources.list()
    resource_list = []

    # Collect metadata for each resource
    for resource in resources:
        resource_id_parts = resource.id.split('/')
        resource_group = resource_id_parts[4] if len(resource_id_parts) > 4 else None

        # Convert properties to a dictionary for more detailed metadata
        properties = resource.additional_properties if hasattr(resource, 'additional_properties') else {}

        # Extract SKU information if available
        sku = None
        if hasattr(resource, 'sku'):
            sku = resource.sku.as_dict() if resource.sku else None

        resource_metadata = {
            'name': resource.name,
            'id': resource.id,
            'type': resource.type,
            'location': resource.location,
            'resource_group': resource_group,
            'tags': resource.tags,
            'properties': properties,  # Include detailed properties here
            'sku': sku  # Include dynamic SKU information
        }
        resource_list.append(resource_metadata)

    return resource_list

if name == "main": fetcher = AzureResourceFetcher() resources_metadata = fetcher.fetch_resources_metadata()

# Output metadata to the console
print(json.dumps(resources_metadata, indent=4))

# Print the total number of resources
print(f"Total number of resources: {len(resources_metadata)}")
---------------
Iam getting the response is  -----

{
    "name": "nbu10143995213919025079snap1722456025_disk_6",
    "id": "/subscriptions/resourceGroups/mli-netbackup-protection-plan-snapshot-central-india/providers/Microsoft.Compute/snapshots/nbu10143995213919025079snap1722456025_disk_6",
    "type": "Microsoft.Compute/snapshots",
    "location": "centralindia",
    "resource_group": "mli-netbackup-protection-plan-snapshot-central-india",
    "tags": {
        "createdby": "cloudpoint"
    },
    "properties": {},
    "sku": {
        "name": "Standard_ZRS",
        "tier": "Standard"
    }
}
I need all resources available SKU response.
msyyc commented 1 month ago

@ChenxiJiang333 please help on this issue

ChenxiJiang333 commented 1 month ago

Hi @lokesh190496, I have succeeded to get all resources under my subscription with your code. Maybe it's just because your client app does not have the get permission on the other resources. You can add such permission to your app in a subscription scope and try it again.

lokesh190496 commented 1 month ago

Hi @lokesh190496, I have succeeded to get all resources under my subscription with your code. Maybe it's just because your client app does not have the get permission on the other resources. You can add such permission to your app in a subscription scope and try it again. @ChenxiJiang333 , When I made the API call for each service separately, I received the expected response.

ChenxiJiang333 commented 1 month ago

Hi @lokesh190496, I have succeeded to get all resources under my subscription with your code. Maybe it's just because your client app does not have the get permission on the other resources. You can add such permission to your app in a subscription scope and try it again. @ChenxiJiang333 , When I made the API call for each service separately, I received the expected response.

Hi, take a try to run this code below before you try to list all the resources.

from azure.identity import ClientSecretCredential
from azure.mgmt.authorization import AuthorizationManagementClient
import uuid
...
client = AuthorizationManagementClient(
        credential=ClientSecretCredential(...),
        subscription_id="SUBSCRIPTION_ID",
)

response = client.role_assignments.create(
    scope="subscriptions/SUBSCRIPTION_ID",
    role_assignment_name=str(uuid.uuid4()),
    parameters={
        "properties": {
            "principalId": CLIENT_ID,
            "principalType": "ServicePrincipal",
            "roleDefinitionId": "/subscriptions/SUBSCRIPTION_ID/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7",
        }
    },
)
github-actions[bot] commented 1 month ago

Hi @lokesh190496. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

github-actions[bot] commented 3 weeks ago

Hi @lokesh190496, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!