grafana / azure-monitor-datasource

Grafana data source for Azure Monitor/Application Insights (deprecated - now included in core Grafana)
Apache License 2.0
92 stars 29 forks source link

Azure Monitor Data Source For Grafana (deprecated - now included in core Grafana)

This codebase has been moved into Grafana since version 6.0 of Grafana - announcement here.

Documentation for Azure Monitor in Grafana

Azure Monitor is the platform service that provides a single source for monitoring Azure resources. Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms and can be used to monitor your live web application - it will automatically detect performance anomalies.

The Azure Monitor Data Source plugin supports Azure Monitor, Azure Log Analytics and Application Insights metrics in Grafana.

Features

Installation

This plugin requires Grafana 4.5.0 or newer (5.2.0 or newer if you are querying Azure Log Analytics).

Grafana Cloud

If you do not have a Grafana Cloud account, you can sign up for one here.

  1. Click on the Install Now button on the Azure Monitor page on Grafana.com. This will automatically add the plugin to your Grafana instance. It might take up to 30 seconds to install. GrafanaCloud Install

  2. Login to your Hosted Grafana instance (go to your instances page in your profile): https://grafana.com/orgs/<yourUserName>/instances/ and the Azure Monitor data source will be installed.

Installation Instructions on the Grafana Docs Site

Docker

  1. Fetch the latest version of grafana from Docker Hub: docker pull grafana/grafana:latest
  2. Run Grafana and install the Azure Monitor plugin with this command:
    docker run -d --name=grafana -p 3000:3000 -e "GF_INSTALL_PLUGINS=grafana-azure-monitor-datasource" grafana/grafana:latest
  3. Open the browser at: http://localhost:3000 or http://your-domain-name:3000
  4. Login in with username: admin and password: admin
  5. To make sure the plugin was installed, check the list of installed data sources. Click the Plugins item in the main menu. Both core data sources and installed data sources will appear.

This ia an alternative command if you want to run Grafana on a different port than the default 3000 port:

docker run -d --name=grafana -p 8081:8081 -e "GF_SERVER_HTTP_PORT=8081" -e "GF_INSTALL_PLUGINS=grafana-azure-monitor-datasource" grafana/grafana:master

It is recommended that you use a volume to save the Grafana data in. Otherwise if you remove the docker container, you will lose all your Grafana data (dashboards, users etc.). You can create a volume with the Docker Volume Driver for Azure File Storage.

Installing the Plugin on an Existing Grafana with the CLI

Grafana comes with a command line tool that can be used to install plugins.

  1. Upgrade Grafana to the latest version. Get that here.
  2. Run this command: grafana-cli plugins install grafana-azure-monitor-datasource
  3. Restart the Grafana server.
  4. Open the browser at: http://localhost:3000 or http://your-domain-name:3000
  5. Login in with a user that has admin rights. This is needed to create data sources.
  6. To make sure the plugin was installed, check the list of installed data sources. Click the Plugins item in the main menu. Both core data sources and installed data sources will appear.

Installing the Plugin Manually on an Existing Grafana

If the server where Grafana is installed has no access to the Grafana.com server, then the plugin can be downloaded and manually copied to the server.

  1. Upgrade Grafana to the latest version. Get that here.
  2. Get the zip file from Grafana.com: https://grafana.com/plugins/grafana-azure-monitor-datasource/installation and click on the link in step 1 (with this text: "Alternatively, you can manually download the .zip file")
  3. Extract the zip file into the data/plugins subdirectory for Grafana.
  4. Restart the Grafana server
  5. To make sure the plugin was installed, check the list of installed data sources. Click the Plugins item in the main menu. Both core data sources and installed data sources will appear.

Configure the data source

The plugin can access metrics from both the Azure Monitor service and the Application Insights API. You can configure access to one service or both services.

  1. Accessed from the Grafana main menu, newly installed data sources can be added immediately within the Data Sources section. Next, click the "Add data source" button in the upper right. The data source will be available for selection in the Type select box.

  2. Select Azure Monitor from the Type dropdown: Data Source Type

  3. In the name field, fill in a name for the data source. It can be anything. Some suggestions are Azure Monitor or App Insights.

  4. If you are using Azure Monitor, then you need 4 pieces of information from the Azure portal (see link above for detailed instructions):

    • Tenant Id (Azure Active Directory -> Properties -> Directory ID)
    • Subscription Id (Subscriptions -> Choose subscription -> Overview -> Subscription ID)
    • Client Id (Azure Active Directory -> App Registrations -> Choose your app -> Application ID)
    • Client Secret ( Azure Active Directory -> App Registrations -> Choose your app -> Keys)
  5. Paste these four items into the fields in the Azure Monitor API Details section: Azure Monitor API Details

  6. If you are also using the Azure Log Analytics service, then you need to specify these two config values (or you can reuse the Client Id and Secret from the previous step).

    • Client Id (Azure Active Directory -> App Registrations -> Choose your app -> Application ID)
    • Client Secret ( Azure Active Directory -> App Registrations -> Choose your app -> Keys -> Create a key -> Use client secret)
  7. If you are are using Application Insights, then you need two pieces of information from the Azure Portal (see link above for detailed instructions):

    • Application ID
    • API Key
  8. Paste these two items into the appropriate fields in the Application Insights API Details section: Application Insights API Details

  9. Test that the configuration details are correct by clicking on the "Save & Test" button: Azure Monitor API Details

Alternatively on step 4 if creating a new Azure Active Directory App, use the Azure CLI:

az ad sp create-for-rbac -n "http://localhost:3000"

Formatting Legend Keys with Aliases

The default legend formatting for the Azure Monitor API is:

resourceName{dimensionValue=dimensionName}.metricName

and for the Application Insights API is:

metric/name{group/by="groupbyvalue"}

These can be quite long but this formatting can be changed using aliases. In the Legend Format field, the aliases which are defined below can be combined any way you want.

Azure Monitor Examples:

Application Insights Examples:

Alias Patterns for Application Insights

Alias Patterns for Azure Monitor

Filter Expressions for Application Insights

The filter field takes an OData filter expression.

Examples:

Azure Log Analytics

Queries are written in the new Azure Log Analytics (or KustoDB) Query Language. A Log Analytics Query can be formatted as Time Series data or as Table data.

Time Series queries are for the Graph Panel (and other panels like the Single Stat panel) and must contain a datetime column, a metric name column and a value column. Here is an example query that returns the aggregated count grouped by the Category column and grouped by hour:

AzureActivity
| where $__timeFilter(TimeGenerated)
| summarize count() by Category, bin(TimeGenerated, 1h)
| order by TimeGenerated asc

Table queries are mainly used in the Table panel and row a list of columns and rows. This example query returns rows with the 6 specified columns:

AzureActivity 
| where $__timeFilter()
| project TimeGenerated, ResourceGroup, Category, OperationName, ActivityStatus, Caller
| order by TimeGenerated desc

Azure Log Analytics Macros

To make writing queries easier there are two Grafana macros that can be used in the where clause of a query:

Azure Log Analytics Builtin Variables

There are also some Grafana variables that can be used in Azure Log Analytics queries:

Writing Analytics Queries For Application Insights

If you change the service type to "Application Insights", the menu icon to the right adds another option, "Toggle Edit Mode". Once clicked, the query edit mode changes to give you a full text area in which to write log analytics queries. (This is identical to how the InfluxDB datasource lets you write raw queries.)

Once a query is written, the column names are automatically parsed out of the response data. You can then select them in the "X-axis", "Y-axis", and "Split On" dropdown menus, or just type them out.

There are some important caveats to remember:

Templating with Variables

Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place. Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data being displayed in your dashboard.

The Azure Monitor Datasource Plugin provides the following queries you can specify in the Query field in the Variable edit view. They allow you to fill a variable's options list.

Application Insights

Name Description
AppInsightsMetricNames() Returns a list of metric names.
AppInsightsGroupBys(aMetricName) Returns a list of group bys for the specified metric name.

Examples:

Azure Monitor

Name Description
ResourceGroups() Returns a list of resource groups.
Namespaces(aResourceGroup) Returns a list of namespaces for the specified resource group.
ResourceNames(aResourceGroup, aNamespace) Returns a list of resource names.
MetricNames(aResourceGroup, aNamespace, aResourceName) Returns a list of metric names.

Examples:

Development

To install and build the plugin:

  1. git clone this project into your data/plugins subdirectory in your Grafana instance.
  2. yarn install --pure-lockfile
  3. grunt
  4. karma start --single-run to run the tests once.
  5. Restart your Grafana server to start using the plugin in Grafana (Grafana only needs to be restarted once).

grunt watch will build the TypeScript files and copy everything to the dist directory automatically when a file changes. This is useful for when working on the code. karma start will turn on the karma file watcher so that it reruns all the tests automatically when a file changes.

The plugin is written in TypeScript and changes should be made in the src directory. The build task transpiles the TypeScript code into JavaScript and copies it to the dist directory. Grafana will load the JavaScript from the dist directory and ignore the src directory.

CHANGELOG

v0.3.0

v0.2.2

v0.2.1

v0.2.0

v0.1.2

Fix for long metric names which means that the Azure WebApp Slots metrics now work. #27

v0.1.1

Small bugfix for the query editor when adding a new panel.

v0.1.0

v0.0.9

v0.0.8

v0.0.7

v0.0.6

v0.0.5

v0.0.4

v0.0.3

Uses the latest version of the Azure Monitor REST API (2017-05-01-preview). Does not currently change anything for the user but enables new features in the future.

v0.0.2

v0.0.1