Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.6k stars 2.81k forks source link

When querying metrics using Azure Monitor Query, it doesn't support interval of less than 1 minute #20327

Closed esttenorio closed 3 years ago

esttenorio commented 3 years ago

We are using the Azure Monitor Query client library to fetch both Logs and Metrics to create a custom monitoring Dashboard.

For logs, we have no issues since the interval can be defined with by bin(TimeGenerated, 10s) in the Kusto query. But for metrics, our team has developed some custom Telegraf plugins that are sending metrics to specific resources. When using the Monitor Query lib, the lowest interval that can be defined is 1m even when the plugin can emit data up to every 10s. We can confirm this by using the Chronograf InfluxDB local ui:

image

image

Since Azure Monitoring does not provide a way to visualize Azure Metrics, using this lib is the only way we've found to do extract metrics and show them next to some other data from Azure Monitor Logs. We are using the lib to get live monitoring of remote machines and we do require smaller intervals to detect "jitter", abnormal fluctuations, and spikes. With the 1m interval, all the spikes and fluctuations are averaged over a minute and are not shown on the output data.

For our application, 5s or 10s would be ideal. Is it possible to remove the 1m limitation and support smaller intervals?


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

xiangyan99 commented 3 years ago

Thanks for the feedback, we’ll investigate asap.

rakshith91 commented 3 years ago

Hi @esttenorio , an interval of less than 1 minute is not supported by the service team currently.

Unfortunately, the backend data store only stores data at 1 minute resolutions and the only way to accomplish this would be to use Kusto queries over log data.

Even if metric data is being sent every second, the backend will be aggregating and rolling it up into 1 minute buckets so that data is lost there is no way to query MDM for less than 1 minute intervals.

we are planning to improve the error message for this situation and tracking it here https://msazure.visualstudio.com/One/_workitems/edit/10693931

ghost commented 3 years ago

Hi @esttenorio. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

ghost commented 3 years ago

Hi @esttenorio, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.