Unable to run Kusto control commands in Panel queries

[ms-anand]

Getting "unsupported analytics column type" error when trying to run Kusto Management (control commands) from a dashboard panel: This used to work in previous versions (3.0.x)

[avosdewael]

I am getting a similar error when trying to run a .show database DatabaseName schema Kusto command in a panel query in version 3.1.0, running on Grafana version 7.2.0. The error's wording is slightly different: Failed to execute query: error converting response to data frames: unsupported analytics column type.

[mckn]

@kylebrandt could you have a look at this? looks like we are not mapping all response data into the data frame properly. I tried running the query with all formats but gets errors in all scenarios.

[kylebrandt]

@mckn This because Data explorer does not properly return ColumnType with this command https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/rest/response#json-encoding-of-a-sequence-of-tables , but not sure where one can file an bug for data explorer side issues?

[erikbos]

I've got a few contacts in the Data Explorer team, they are quite good with followups. If you could post an example response where this happens than I'll open a bug with them.

[kylebrandt]

I've got a few contacts in the Data Explorer team, they are quite good with followups. If you could post an example response where this happens than I'll open a bug with them.

With the query: .show table Covid19 details | project format_bytes(TotalOriginalSize, 2)

results are:

{
  "Tables": [
    {
      "TableName": "Table_0",
      "Columns": [
        {
          "ColumnName": "Column1",
          "DataType": "String"
        }
      ],
      "Rows": [
        [
          "5.78 MB"
        ]
      ]
    },
    {
      "TableName": "Table_1",
      "Columns": [
        {
          "ColumnName": "Value",
          "DataType": "String"
        }
      ],
      "Rows": [
        [
          "{\"Visualization\":null,\"Title\":null,\"XColumn\":null,\"Series\":null,\"YColumns\":null,\"AnomalyColumns\":null,\"XTitle\":null,\"YTitle\":null,\"XAxis\":null,\"YAxis\":null,\"Legend\":null,\"YSplit\":null,\"Accumulate\":false,\"IsQuerySorted\":false,\"Kind\":null,\"Ymin\":\"NaN\",\"Ymax\":\"NaN\"}"
        ]
      ]
    },
    {
      "TableName": "Table_2",
      "Columns": [
        {
          "ColumnName": "Timestamp",
          "DataType": "DateTime"
        },
        {
          "ColumnName": "Severity",
          "DataType": "Int32"
        },
        {
          "ColumnName": "SeverityName",
          "DataType": "String"
        },
        {
          "ColumnName": "StatusCode",
          "DataType": "Int32"
        },
        {
          "ColumnName": "StatusDescription",
          "DataType": "String"
        },
        {
          "ColumnName": "Count",
          "DataType": "Int32"
        },
        {
          "ColumnName": "RequestId",
          "DataType": "Guid"
        },
        {
          "ColumnName": "ActivityId",
          "DataType": "Guid"
        },
        {
          "ColumnName": "SubActivityId",
          "DataType": "Guid"
        },
        {
          "ColumnName": "ClientActivityId",
          "DataType": "String"
        }
      ],
      "Rows": [
        [
          "2020-10-07T16:32:07.4143908Z",
          4,
          "Info",
          0,
          "Query completed successfully",
          1,
          "94c2d596-8506-4855-8f78-fcc9142f95c5",
          "94c2d596-8506-4855-8f78-fcc9142f95c5",
          "cd9b7679-89c8-4fab-a5bc-0d1c2c68f50b",
          "KGC.raw;5a06ddc0-b8de-413a-98d6-57ec324c8e9a"
        ],
        [
          "2020-10-07T16:32:07.4143908Z",
          6,
          "Stats",
          0,
          "{\"ExecutionTime\":0.0,\"resource_usage\":{\"cache\":{\"memory\":{\"hits\":0,\"misses\":0,\"total\":0},\"disk\":{\"hits\":0,\"misses\":0,\"total\":0},\"shards\":{\"hot\":{\"hitbytes\":0,\"missbytes\":0,\"retrievebytes\":0},\"cold\":{\"hitbytes\":0,\"missbytes\":0,\"retrievebytes\":0},\"bypassbytes\":0}},\"cpu\":{\"user\":\"00:00:00\",\"kernel\":\"00:00:00\",\"total cpu\":\"00:00:00\"},\"memory\":{\"peak_per_node\":0}},\"input_dataset_statistics\":{\"extents\":{\"total\":0,\"scanned\":0,\"scanned_min_datetime\":\"0001-01-01T00:00:00.0000000Z\",\"scanned_max_datetime\":\"0001-01-01T00:00:00.0000000Z\"},\"rows\":{\"total\":0,\"scanned\":0},\"rowstores\":{\"scanned_rows\":0,\"scanned_values_size\":0},\"shards\":{\"queries_generic\":0,\"queries_specialized\":0}},\"dataset_statistics\":[{\"table_row_count\":1,\"table_size\":16}]}",
          1,
          "94c2d596-8506-4855-8f78-fcc9142f95c5",
          "94c2d596-8506-4855-8f78-fcc9142f95c5",
          "cd9b7679-89c8-4fab-a5bc-0d1c2c68f50b",
          "KGC.raw;5a06ddc0-b8de-413a-98d6-57ec324c8e9a"
        ]
      ]
    },
    {
      "TableName": "Table_3",
      "Columns": [
        {
          "ColumnName": "Ordinal",
          "DataType": "Int64"
        },
        {
          "ColumnName": "Kind",
          "DataType": "String"
        },
        {
          "ColumnName": "Name",
          "DataType": "String"
        },
        {
          "ColumnName": "Id",
          "DataType": "String"
        },
        {
          "ColumnName": "PrettyName",
          "DataType": "String"
        }
      ],
      "Rows": [
        [
          0,
          "QueryResult",
          "PrimaryResult",
          "74d8591f-0430-43ac-a283-8ecdd3bd8c49",
          ""
        ],
        [
          1,
          "QueryProperties",
          "@ExtendedProperties",
          "45659004-6092-45e6-bc10-62588ec695ff",
          ""
        ],
        [
          2,
          "QueryStatus",
          "QueryStatus",
          "00000000-0000-0000-0000-000000000000",
          ""
        ]
      ]
    }
  ]
}

So it is sending the .NET type in this case DataType but is missing kusto ColumnType. I'm not to keen on adding behavior to the plugin to handle this alternate response format since:

1. I don't see this behavior documented, so not sure what is "correct" - are there times that there might be ColumnType and not DataType, are there case where there are both? If there are both - can they conflict? If so, which one is right?
2. The v2 API only has ColumnType, so if we switch to that this behavior would not be relevant.

[erikbos]

Thanks for details, I've relayed it to ADX team.

[erikbos]

My contact was on holiday last week, case to investigate is now open: 120101422000599

[aengusrooneygrafana]

Hi @ms-anand could you please check for any updates on case 120101422000599 and share the updates here please? Thanks!

[zivcaspi]

Thanks for letting us know -- this is a bug (an admin command piped through to a query "forgets" to write the appropriate ColumnType property in the result). We have a repro of the issue and are working on a fix.

[mckn]

@zivcaspi would be awesome if you could keep us posted when this is resolved.

[avosdewael]

It appears Microsoft has solved the issue, as I can no longer reproduce it. Grafana now correctly loads the output from ADX control commands from my ADX cluster. However, I do not know whether this fix has yet been deployed to all ADX clusters everywhere.

@zivcaspi could you please confirm the happy news?

[joshhunt]

Going to close this one out - let us know if this is still happening!