Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.53k stars 2.76k forks source link

Need instructions for enabling diagnostic settings for Entra ID using MonitorManagementClient from azure.mgmt.monitor. #36776

Open Shantanudevil opened 1 month ago

Shantanudevil commented 1 month ago

Type of issue

Code doesn't work

Description

I am willing to enable diagnostic settings for Entra ID & forward those logs to an EventHub. However, the existing code available on the internet is not functional.

Page URL

https://learn.microsoft.com/en-us/python/api/azure-mgmt-monitor/azure.mgmt.monitor.monitormanagementclient?view=azure-python

Content source URL

https://github.com/MicrosoftDocs/azure-docs-sdk-python/blob/main/docs-ref-autogen/azure-mgmt-monitor/azure.mgmt.monitor.MonitorManagementClient.yml

Document Version Independent Id

ee43dd9e-a405-9abb-eda9-9d2fc6494066

Article author

@lmazuel

Metadata

github-actions[bot] commented 1 month ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

github-actions[bot] commented 1 month ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @gracewilcox @gulopesd @Haiying-MSFT @jairmyree @joshfree @KarishmaGhiya @KevinBlasko @kurtzeborn @nisha-bhatia @pvaneck @sarangan12 @scottaddie @srnagar @ToddKingMSFT.

pvaneck commented 1 month ago

Hey @Shantanudevil, do you mind providing a code snippet along with what errors you are encountering using the azure-mgmt-monitor library?

github-actions[bot] commented 1 month ago

Hi @Shantanudevil. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

msyyc commented 1 month ago

Add @ChenxiJiang333 for help

Shantanudevil commented 1 month ago

@pvaneck - I am not able to understand what to pass as resource URI for Entra ID diagnostic settings available, additionally defining the log categories that needs to be enabled.

Below is the attached snip of code which is used for enabling diagnostic settings for subscription.

image

Shantanudevil commented 1 month ago

@pvaneck / @msyyc / @ChenxiJiang333 any prompt help on this would be really appreciated.

ChenxiJiang333 commented 1 month ago

@pvaneck - I am not able to understand what to pass as resource URI for Entra ID diagnostic settings available, additionally defining the log categories that needs to be enabled.

Below is the attached snip of code which is used for enabling diagnostic settings for subscription.

image

Hi, not sure whether such operation can be done in a tenant scope, maybe you can try pass the resource_uri with value /tenants/<TENANT ID>/providers/Microsoft.aadiam.

Shantanudevil commented 1 month ago

@ChenxiJiang333 - it's not working throwing below error -

image

Parameters -

image

Would you please be able to let me know the correct syntax / function code to enable diagnostic settings with a category of logs being forwarded to an EventHub? as I have been performing guesswork for a long time now, with no success.

pvaneck commented 1 month ago

I believe for Azure Monitor, diagnostics settings can only be applied to specific resources and not the subscription itself. Resource URIs (also known as Resource IDs) are of the format:

/subscriptions/{guid}/resourceGroups/{resource-group-name}/{resource-provider-namespace}/{resource-type}/{resource-name}`

You'll likely have to apply diagnostic settings to each resource of interest.

A sample I found for this specific method is here, which may help.

If you need to find a list of resources in a subscription, then you can use the azure-mgmt-resource package:

from azure.identity import DefaultAzureCredential
from azure.mgmt.resource import ResourceManagementClient

subscription_id = 'your-subscription-id'
credential = DefaultAzureCredential()
resource_client = ResourceManagementClient(credential, subscription_id)

# List all resources in the subscription
resources = resource_client.resources.list()

for resource in resources:
    # Do something with resource.id

Let me know if this helps.

@ChenxiJiang333, I think it would be nice to update the resource_uri docstring to include the expected format similar to: https://learn.microsoft.com/en-us/rest/api/resources/resources/get-by-id?view=rest-resources-2021-04-01#uri-parameters

github-actions[bot] commented 1 month ago

Hi @Shantanudevil. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

ChenxiJiang333 commented 1 month ago

I believe for Azure Monitor, diagnostics settings can only be applied to specific resources and not the subscription itself. Resource URIs (also known as Resource IDs) are of the format:

/subscriptions/{guid}/resourceGroups/{resource-group-name}/{resource-provider-namespace}/{resource-type}/{resource-name}`

You'll likely have to apply diagnostic settings to each resource of interest.

A sample I found for this specific method is here, which may help.

If you need to find a list of resources in a subscription, then you can use the azure-mgmt-resource package:

from azure.identity import DefaultAzureCredential
from azure.mgmt.resource import ResourceManagementClient

subscription_id = 'your-subscription-id'
credential = DefaultAzureCredential()
resource_client = ResourceManagementClient(credential, subscription_id)

# List all resources in the subscription
resources = resource_client.resources.list()

for resource in resources:
    # Do something with resource.id

Let me know if this helps.

@ChenxiJiang333, I think it would be nice to update the resource_uri docstring to include the expected format similar to: https://learn.microsoft.com/en-us/rest/api/resources/resources/get-by-id?view=rest-resources-2021-04-01#uri-parameters

The docstring was defined in swagger, has opened an issue to report it. https://github.com/Azure/azure-rest-api-specs/issues/30116

Shantanudevil commented 1 month ago

Hi @ChenxiJiang333 ,

I want to enable diagnostic settings for Entra Tenant instead of a specific resource with the requirement being a certain category of logs would be forwarded to an EventHub. Hope I am clear on the requirements.

Shantanudevil commented 1 month ago

@msyyc / @ChenxiJiang333 / @msyyc any prompt help on this would be really appreciated.

Shantanudevil commented 1 month ago

Any help? >﹏< @pvaneck / @msyyc / @ChenxiJiang333

pvaneck commented 1 month ago

Hey, sorry. I'm not really all that familiar with this SDK, so my knowledge is very limited here, and unfortunately, I don't currently have the means to test tenant-level operations.

I did find this issue: https://github.com/Azure/azure-sdk-for-python/issues/18476 which seems to indicate that this scenario isn't supported, and instead, a user has to send REST requests manually to the https://management.azure.com/providers/microsoft.aadiam/diagnosticSettings/ endpoint.

I see some blogs showcasing this usage like here and here.

@ChenxiJiang333 @msyyc Do you happen to know any more about a client for these tenant-level diagnostic settings management operations?

msyyc commented 1 month ago

SDK is generated with swagger but I am not from service team so don't have enough domain knowledge about specific usage scenario of specific azure service. @Shantanudevil I advise you open an issue in https://github.com/Azure/azure-rest-api-specs/issues to seek help from service team about whether there is REST API for your required scenario. Once you find the REST API, we could help you find the related SDK API.

Shantanudevil commented 1 month ago

Hi @msyyc - I can see a related issue already opened - https://github.com/Azure/azure-rest-api-specs/issues/11085. Wehre they have mentioned the rest API available i.e., - PUT /providers/microsoft.aadiam/diagnosticSettings/testDiagSetting?api-version=2017-04-01.

Would request you to please let me know the related SDK for this.

msyyc commented 1 month ago

This REST API is a little old. Could you please try if this API could meet your requirements? If yes, I can contact service team whether this REST API is ok to release as SDK API.

Shantanudevil commented 1 month ago

yeah, it is working for us as of now. It would be really great if a SDK can be released as we are willing to stick to the python SDK instead of using request library with REST Api.

Additionally, is there any expected ETA or time delay for the release of SDK ?

Shantanudevil commented 4 weeks ago

Hi @msyyc - any update on this ?

msyyc commented 3 weeks ago

@Shantanudevil Your wanted API is in https://github.com/Azure/azure-rest-api-specs/tree/main/specification/azureactivedirectory/resource-manager which is not updated for long time and I can't find who to contact. My colleague tells me that https://learn.microsoft.com/en-us/graph/overview may have similar function.

github-actions[bot] commented 3 weeks ago

Hi @Shantanudevil. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

Shantanudevil commented 3 weeks ago

Apologies @msyyc, I am really not sure if Microsoft Graph is really relevant here i.e., For Diagnostic settings for Entra tenant.

Additionally, I do not find - https://github.com/Azure/azure-rest-api-specs/tree/main/specification/azureactivedirectory/resource-manager helpful. Would you be able to share a code snippet example? or import statements with the specific function to enable the same?

msyyc commented 3 weeks ago

image This rest API is defined in https://github.com/Azure/azure-rest-api-specs/blob/48dcf5d28914f6d5ba2d66aa1689241b29bac49d/specification/azureactivedirectory/resource-manager/Microsoft.Aadiam/stable/2017-04-01/azureactivedirectory.json#L106-L183. However this folder is not updated for years and I can't find member of service team to confirm whether this API is still recommended to use.

Shantanudevil commented 3 weeks ago

@msyyc - would you please be able to confirm on the way ahead on this? or whom should I reach out to get this expediated?

msyyc commented 3 weeks ago

I advise you open an issue in https://github.com/Azure/azure-rest-api-specs/issues to confirm whether https://github.com/Azure/azure-rest-api-specs/blob/48dcf5d28914f6d5ba2d66aa1689241b29bac49d/specification/azureactivedirectory/resource-manager/Microsoft.Aadiam/stable/2017-04-01/azureactivedirectory.json is still maintained. If there is comment from a member of service team, we can contact them to release SDK if possible.

Kaloszer commented 2 weeks ago

@msyyc this is already there and not touched by anyone for ages now https://github.com/Azure/azure-rest-api-specs/issues/11085

msyyc commented 2 weeks ago

It proves that this service may be not maintained actively so SDK team won't release SDK for it. I am afraid you have to find another service to replace it.

Kaloszer commented 2 weeks ago

@msyyc I mean, it's used in the Azure Entra Data Connector in Microsoft Sentinel which is a Microsoft solution connector, so it's not like I can look for another service to replace it 🤣. This missing just forces me to do click-ops

msyyc commented 2 weeks ago

Is it possible to use private package? Since our SDK is also generated from swagger with codegen tool, the content of public SDK actually same with private package. And here is the guidance of how to get private package by yourself: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/private_package/get_private_package.md

github-actions[bot] commented 6 days ago

Hi @Shantanudevil. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.