grafana / azure-data-explorer-datasource

Grafana datasource plugin for Microsoft Azure Data Explorer (formerly Azure Kusto)
Apache License 2.0
49 stars 39 forks source link

Support for empty or missing `database` key in dashboard JSON, fallback to default #615

Closed kspeeckaert closed 5 months ago

kspeeckaert commented 1 year ago

When using an ADX data source in a dashboard, the underlying dashboard JSON will look something like this:

{
  "panels": [
    {
      "datasource": {
        "type": "grafana-azure-data-explorer-datasource",
        "uid": "cfb97634-30e4-4c76-a9ef-ddbe79be15d7"
      },
      "targets": [
        {
          "OpenAI": false,
          "database": "demo_db",
          "datasource": {
            "type": "grafana-azure-data-explorer-datasource",
            "uid": "cfb97634-30e4-4c76-a9ef-ddbe79be15d7"
          },
...

Note that the database is referenced. This complicates provisioning dashboards, as we can't reference environment variables from JSON files (only from the provisioning config files), and we're stuck with a hardcoded database name.

To give some context: we spin up a separate Grafana instance per customer, and each instance is tied to the same storage for provisioning. Everything that's customer-specific is set through environment variables. Databases are also customer-specific, and this works great for provisioning config files (e.g. to create the data sources), but breaks in dashboards where ADX is used. In contrast, the MSSQL plugin has no issue.

As we already defined the database in the data source definition, there is no point in replicating it again here. If the database is the same as the default database (i.e. the one in the data source definition), the database entry should be blank or missing.

If that isn't possible, it should at least be possible to remove or blank the database entry in the dashboard JSON files for provisioning, and at runtime, a missing or blank database should be replaced by the default one. Alas, if we do this, this results in an error:

Azure HTTP "520 InternalServiceError": "name (Parameter 'Argument 'name' is empty: at .ctor in C:\\__w\\1\\s\\Src\\Engine\\DataNode\\Common\\Entities\\DatabaseIdentifier.cs: line 88')\r\nTimestamp=2023-09-01T06:52:26.9753150Z\r\nClientRequestId=KGC.raw;a4f21088ab4ff7d6;admin\r\nActivityId=REDACTED-GUID\r\nActivityType=GW.Http.CallContext\r\nServiceAlias=REDACTED\r\nMachineName=KEngine000000\r\nProcessName=Kusto.WinSvc.Svc\r\nProcessId=9212\r\nThreadId=8620\r\nActivityStack=(Activity stack: CRID=KGC.raw;a4f21088ab4ff7d6;admin ARID=REDACTED-GUID> GW.Http.CallContext/REDACTED-GUID)\r\nMonitoredActivityContext=(ActivityType=GW.Http.CallContext, Timestamp=2023-09-01T06:52:26.9701541Z, ParentActivityId=REDACTED-GUID, TimeSinceStarted=5.1904 [ms])"

Environment:

aangelisc commented 1 year ago

Hi @kspeeckaert, thank you for raising this. We'll investigate how we can approach resolving this 😊