[BUG] Query Execute Error when creating Per Document Monitor

[kavilla]

@waza-ari commented on Fri Apr 21 2023

Creating Document Level monitor fails as the query field value is not passed correctly

A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. You need to have at least one index created
2. Go to Alerting -> Monitors -> Create Monitor
3. Enter any name
4. Select Per Document Monitor
5. Select any index
6. Add Any name for query and add any field and value
7. Dashboards will not send a POST request to /api/alerting/monitors/_execute which will cause an uncaught exception on the opensearch node. As a result, we cannot proceed creating the monitor, as the requests somehow "clog" the opensearch node and the notification channels can't be loaded

If we wait roughly 5 minutes, it will somehow clear up and the notification channels actually get loaded. We can then proceed with creation and the monitor works fine.

Expected behavior The query should be created normally and the available notification channels should be shown. Works for Per Query Monitors for example

OpenSearch Version 2.6.0

Dashboards Version 2.6.0

Plugins

Default configuration

Screenshots

If applicable, add screenshots to help explain your problem.

This will yield this error message even though a field value is set:

Host/Environment (please complete the following information):

• OS: Server: Ubuntu 22.04 using Docker, Client: Mac OS X
• Browser and version: Safari 16.4 and Firefox 112.0.1 (same behaviour)

Additional context

Add any other context about the problem here.

The error message shown on the opensearch node:

opensearch-node1  | [2023-04-21T13:13:56,907][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [opensearch-node1] uncaught exception in thread [DefaultDispatcher-worker-1]
opensearch-node1  | java.lang.NullPointerException: null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>
opensearch-node1  |     at org.opensearch.alerting.util.DocLevelMonitorQueries.traverseMappingsAndUpdate(DocLevelMonitorQueries.kt:183) ~[opensearch-alerting-2.6.0.0.jar:2.6.0.0]
opensearch-node1  |     at org.opensearch.alerting.util.DocLevelMonitorQueries.indexDocLevelQueries(DocLevelMonitorQueries.kt:245) ~[opensearch-alerting-2.6.0.0.jar:2.6.0.0]
opensearch-node1  |     at org.opensearch.alerting.transport.TransportExecuteMonitorAction$doExecute$1$2.invokeSuspend(TransportExecuteMonitorAction.kt:134) ~[opensearch-alerting-2.6.0.0.jar:2.6.0.0]
opensearch-node1  |     at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1  |     at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |     at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |     at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |     at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |     Suppressed: java.lang.NullPointerException: null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>
opensearch-node1  |         at org.opensearch.alerting.util.DocLevelMonitorQueries.traverseMappingsAndUpdate(DocLevelMonitorQueries.kt:183) ~[opensearch-alerting-2.6.0.0.jar:2.6.0.0]
opensearch-node1  |         at org.opensearch.alerting.util.DocLevelMonitorQueries.indexDocLevelQueries(DocLevelMonitorQueries.kt:245) ~[opensearch-alerting-2.6.0.0.jar:2.6.0.0]
opensearch-node1  |         at org.opensearch.alerting.transport.TransportExecuteMonitorAction$doExecute$1$2.invokeSuspend(TransportExecuteMonitorAction.kt:134) ~[opensearch-alerting-2.6.0.0.jar:2.6.0.0]
opensearch-node1  |         at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1  |         at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |         at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |         at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  |         at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1  | java.lang.NullPointerException: null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>
opensearch-node1  |     at org.opensearch.alerting.util.DocLevelMonitorQueries.traverseMappingsAndUpdate(DocLevelMonitorQueries.kt:183)
opensearch-node1  |     at org.opensearch.alerting.util.DocLevelMonitorQueries.indexDocLevelQueries(DocLevelMonitorQueries.kt:245)
opensearch-node1  |     at org.opensearch.alerting.transport.TransportExecuteMonitorAction$doExecute$1$2.invokeSuspend(TransportExecuteMonitorAction.kt:134)
opensearch-node1  |     at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
opensearch-node1  |     at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233)
opensearch-node1  |     at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
opensearch-node1  |     at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
opensearch-node1  |     at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)
opensearch-node1  |     Suppressed: java.lang.NullPointerException: null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>
opensearch-node1  |         ... 8 more

JSON sent to /api/alerting/monitors/_execute:

{
  "name": "TEMP_MONITOR",
  "type": "monitor",
  "monitor_type": "doc_level_monitor",
  "enabled": true,
  "schedule": {
    "period": {
      "interval": 1,
      "unit": "MINUTES"
    }
  },
  "inputs": [
    {
      "doc_level_input": {
        "description": "",
        "indices": [
          "app.backend.app_logs-2023-04-21"
        ],
        "queries": [
          {
            "id": "sev",
            "name": "sev",
            "query": "severity:\"E\"",
            "tags": []
          }
        ]
      }
    }
  ],
  "triggers": [],
  "ui_metadata": {
    "schedule": {
      "timezone": null,
      "frequency": "interval",
      "period": {
        "interval": 1,
        "unit": "MINUTES"
      },
      "daily": 0,
      "weekly": {
        "mon": false,
        "tue": false,
        "wed": false,
        "thur": false,
        "fri": false,
        "sat": false,
        "sun": false
      },
      "monthly": {
        "type": "day",
        "day": 1
      },
      "cronExpression": "0 */1 * * *"
    },
    "monitor_type": "doc_level_monitor",
    "doc_level_input": {
      "queries": [
        {
          "queryName": "sev",
          "field": "severity",
          "operator": "==",
          "query": "E",
          "tags": []
        }
      ]
    },
    "search": {
      "searchType": "graph"
    }
  }
}

[getsaurabh02]

cc: @lezzago

[waza-ari]

This issue is as far as I can tell still here, still throwing issues on the frontend. Is there anything in terms of information that you'd need from me? Happy to help if I can.