Creating index pattern does not work in version 7.16_latest

[marinkepec]

There is an issue while creating Kibana index pattern while using the K2Bridge image version 7.16_latest, and also with the image 0.5.15164 that are the latest images at the moment.

When trying to create an index pattern there is an available list of all the tables on the ADX database, and when trying to create an index pattern for one of the tables there is a message: The indices which match this index pattern don't contain any time fields. But, in the table timestamp as information exists in the field timestamp. The index pattern fails to be created with an error: Error fetching fields for index pattern

Here are some of the screenshots:

From the k2bridge deployment logs, there are some that seem to be produced at the time of failed index pattern creation and that could be useful for troubleshooting the issue. If there are some other logs and information that could be useful, they can also be provided at request.

{"@t":"2023-06-28T10:49:52.8278841Z","@mt":"Translate params: header:{@header}, query:{@query}","@l":"Debug","header":"{\"index\":\"loggingAppDB:GELFLogs*\"}","query":{"Data":"{\"size\":0,\"aggs\":{\"indices\":{\"terms\":{\"field\":\"_index\",\"size\":200}}}}"},"SourceContext":"K2Bridge.ElasticQueryTranslator","ActionId":"b23454a6-cdea-452d-a72d-df33c221fd89","ActionName":"K2Bridge.Controllers.QueryController.SingleSearchAsync (K2Bridge)","RequestId":"0HMRNJV1G883V:00000016","RequestPath":"/loggingAppDB:GELFLogs*/_search","ConnectionId":"0HMRNJV1G883V","CorrelationId":"f2e4d341-2453-4235-ab27-834cc9ce6ad4"}
{"@t":"2023-06-28T10:49:52.8292286Z","@mt":"Failed to execute translate operation.","@l":"Error","@x":"System.ArgumentNullException: Query cannot be null (Parameter 'Query')\n   at K2Bridge.Ensure.ConstructMessageAndThrowArgumentOrNullArgument[T](T arg, String argName, String predefinedMessage, ILogger logger) in /app/K2Bridge/Ensure.cs:line 92\n   at K2Bridge.ElasticQueryTranslator.TranslateQuery(String header, String query) in /app/K2Bridge/ElasticQueryTranslator.cs:line 49","SourceContext":"K2Bridge.ElasticQueryTranslator","ActionId":"b23454a6-cdea-452d-a72d-df33c221fd89","ActionName":"K2Bridge.Controllers.QueryController.SingleSearchAsync (K2Bridge)","RequestId":"0HMRNJV1G883V:00000016","RequestPath":"/loggingAppDB:GELFLogs*/_search","ConnectionId":"0HMRNJV1G883V","CorrelationId":"f2e4d341-2453-4235-ab27-834cc9ce6ad4"}
{"@t":"2023-06-28T10:49:52.8295303Z","@mt":"Failed translating elasticsearch query","@l":"Error","SourceContext":"K2Bridge.Controllers.QueryController","ActionId":"b23454a6-cdea-452d-a72d-df33c221fd89","ActionName":"K2Bridge.Controllers.QueryController.SingleSearchAsync (K2Bridge)","RequestId":"0HMRNJV1G883V:00000016","RequestPath":"/loggingAppDB:GELFLogs*/_search","ConnectionId":"0HMRNJV1G883V","CorrelationId":"f2e4d341-2453-4235-ab27-834cc9ce6ad4"}
{"@t":"2023-06-28T10:52:37.5007653Z","@mt":"Listing tables matching '{@indexName}'","@l":"Debug","indexName":"*:*","SourceContext":"K2Bridge.KustoDAL.KustoDataAccess","ActionId":"1943a0e3-4938-4e84-9026-a2548e0b8332","ActionName":"K2Bridge.Controllers.IndexListController.Resolve (K2Bridge)","RequestId":"0HMRNJV1G885U:00000002","RequestPath":"/_resolve/index/*:*","ConnectionId":"0HMRNJV1G885U","CorrelationId":"fb1293f7-933a-4d7a-94d5-d800bf977afc"}
{"@t":"2023-06-28T10:52:37.5010166Z","@mt":"Calling adminClient.ExecuteControlCommand with the command: {@command}","@l":"Debug","command":{"Data":".show databases schema | where TableName != '' | distinct TableName, DatabaseName | search TableName: '*' | search DatabaseName: '*' |  project strcat(DatabaseName, \":\", TableName)"},"SourceContext":"K2Bridge.KustoDAL.KustoQueryExecutor","ActionId":"1943a0e3-4938-4e84-9026-a2548e0b8332","ActionName":"K2Bridge.Controllers.IndexListController.Resolve (K2Bridge)","RequestId":"0HMRNJV1G885U:00000002","RequestPath":"/_resolve/index/*:*","ConnectionId":"0HMRNJV1G885U","CorrelationId":"fb1293f7-933a-4d7a-94d5-d800bf977afc"}
{"@t":"2023-06-28T10:52:37.5504433Z","@mt":"Translate params: header:{@header}, query:{@query}","@l":"Debug","header":"{\"index\":\"*:*\"}","query":{"Data":"{\"size\":0,\"aggs\":{\"indices\":{\"terms\":{\"field\":\"_index\",\"size\":200}}}}"},"SourceContext":"K2Bridge.ElasticQueryTranslator","ActionId":"b23454a6-cdea-452d-a72d-df33c221fd89","ActionName":"K2Bridge.Controllers.QueryController.SingleSearchAsync (K2Bridge)","RequestId":"0HMRNJV1G883V:00000022","RequestPath":"/*:*/_search","ConnectionId":"0HMRNJV1G883V","CorrelationId":"7b3eb7ca-b770-4157-a3c6-2fec1c09c2eb"}
{"@t":"2023-06-28T10:52:37.5507401Z","@mt":"Failed to execute translate operation.","@l":"Error","@x":"System.ArgumentNullException: Query cannot be null (Parameter 'Query')\n   at K2Bridge.Ensure.ConstructMessageAndThrowArgumentOrNullArgument[T](T arg, String argName, String predefinedMessage, ILogger logger) in /app/K2Bridge/Ensure.cs:line 92\n   at K2Bridge.ElasticQueryTranslator.TranslateQuery(String header, String query) in /app/K2Bridge/ElasticQueryTranslator.cs:line 49","SourceContext":"K2Bridge.ElasticQueryTranslator","ActionId":"b23454a6-cdea-452d-a72d-df33c221fd89","ActionName":"K2Bridge.Controllers.QueryController.SingleSearchAsync (K2Bridge)","RequestId":"0HMRNJV1G883V:00000022","RequestPath":"/*:*/_search","ConnectionId":"0HMRNJV1G883V","CorrelationId":"7b3eb7ca-b770-4157-a3c6-2fec1c09c2eb"}
{"@t":"2023-06-28T10:52:37.5510253Z","@mt":"Failed translating elasticsearch query","@l":"Error","SourceContext":"K2Bridge.Controllers.QueryController","ActionId":"b23454a6-cdea-452d-a72d-df33c221fd89","ActionName":"K2Bridge.Controllers.QueryController.SingleSearchAsync (K2Bridge)","RequestId":"0HMRNJV1G883V:00000022","RequestPath":"/*:*/_search","ConnectionId":"0HMRNJV1G883V","CorrelationId":"7b3eb7ca-b770-4157-a3c6-2fec1c09c2eb"}

[AsafMah]

Hey, I'll need more information about your setup.

Specfically, how you run kibana, the backend elastic, and which versions do they use.

I want to emphasize that we only support the oss versions of the images.