search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.61k stars 8.22k forks source link

Unexpected service crash while rule preview #143595

Closed angorayc closed 1 year ago

angorayc commented 2 years ago

I was looking into an sdh issue complains about threat indicator match rule timeout. I assume the user might have highly match between event list and threat list, so created a rule with an extreme case. The server seems to crash while preview rule. I used very small time range for both preview timeframe and threat indicator time range.

When I tested on my cloud instance, the Kibana hanged: Kibana version: 8.5 latest test video: https://drive.google.com/file/d/1cr3TGLE1X2XwuIQiylhNdqIBSA7Fhz9_/view?usp=sharing

When I tested it on local, my Kibana server hanged: test video: https://drive.google.com/file/d/1JUmjsPEbVveEoEpzE-WiXYXDXWijT2Ab/view?usp=sharing

Data:

  1. auditbeat
  2. filebeat with threat intel module otx enabled. https://www.elastic.co/security-labs/ingesting-threat-data-with-the-threat-intel-filebeat-module
elasticmachine commented 2 years ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 2 years ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

vitaliidm commented 1 year ago

When the number of indicators too large, in this case, around 100k, Kibana crashes when trying to load all results in memory. In test case, that leads to Kibana crash:> Total event count: 861

Total indicator items: 113982

Problematic place is getAllThreatListHits method


export const getAllThreatListHits = async (
  params: Omit<GetThreatListOptions, 'searchAfter'>
): Promise<ThreatListItem[]> => {
  let allThreatListHits: ThreatListItem[] = [];
  let threatList = await getThreatList({ ...params, searchAfter: undefined });

  allThreatListHits = allThreatListHits.concat(threatList.hits.hits);

  while (threatList.hits.hits.length !== 0) {
    threatList = await getThreatList({
      ...params,
      searchAfter: threatList.hits.hits[threatList.hits.hits.length - 1].sort,
    });

    allThreatListHits = allThreatListHits.concat(threatList.hits.hits);
  }
  return allThreatListHits;
};

Although, threats loaded page by page, they get concatenated after each iteration, increases size of allThreatListHits eventually to the point when Kibana process crashes out of memory

 <--- Last few GCs ---> [58116:0x7ff308008000]    64668 ms: Mark-sweep 4014.6 (4141.1) -> 4014.5 (4140.6) MB, 792.5 / 0.1 ms  (average mu = 0.236, current mu = 0.066) allocation failure scavenge might not succeed [58116:0x7ff308008000]    66198 ms: Mark-sweep 4030.3 (4140.6) -> 4030.3 (4172.6) MB, 1514.7 / 0.1 ms  (average mu = 0.096, current mu = 0.010) allocation failure scavenge might not succeed <--- JS stacktrace ---> FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory  1: 0x102b355f5 node::Abort() (.cold.1) [/.nvm/versions/node/v16.18.1/bin/node]  2: 0x10182af49 node::Abort() [/.nvm/versions/node/v16.18.1/bin/node]  3: 0x10182b12e node::OOMErrorHandler(char const, bool) [/.nvm/versions/node/v16.18.1/bin/node]  4: 0x1019a23f0 v8::Utils::ReportOOMFailure(v8::internal::Isolate, char const, bool) [/.nvm/versions/node/v16.18.1/bin/node]  5: 0x1019a23b3 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate, char const, bool) [/.nvm/versions/node/v16.18.1/bin/node]  6: 0x101b46095 v8::internal::Heap::FatalProcessOutOfMemory(char const) [/.nvm/versions/node/v16.18.1/bin/node]  7: 0x101b44a1c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/.nvm/versions/node/v16.18.1/bin/node]  8: 0x101b512c0 v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/.nvm/versions/node/v16.18.1/bin/node]  9: 0x101b51341 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/.nvm/versions/node/v16.18.1/bin/node] 10: 0x101b1832d v8::internal::FactoryBase::NewRawOneByteString(int, v8::internal::AllocationType) [/.nvm/versions/node/v16.18.1/bin/node] 11: 0x101c53cce v8::internal::JsonParser::MakeString(v8::internal::JsonString const&, v8::internal::Handle) [/.nvm/versions/node/v16.18.1/bin/node] 12: 0x101c5267c v8::internal::JsonParser::ParseJsonValue() [/.nvm/versions/node/v16.18.1/bin/node] 13: 0x101c51c42 v8::internal::JsonParser::ParseJson() [/.nvm/versions/node/v16.18.1/bin/node] 14: 0x101a2b18a v8::internal::Builtin_JsonParse(int, unsigned long, v8::internal::Isolate) [/.nvm/versions/node/v16.18.1/bin/node] 15: 0x10227ecf9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/.nvm/versions/node/v16.18.1/bin/node]

Possible way of solving this could be limiting the number of threats loaded, which could potentially lead to non-exhaustive results.

Further, a large number of threats could cause 2 other issues:

  1. In the testing scenario, 100 signals had 2,000 threats. When trying to bulkCreate 100 alerts, the request has failed with 413 code status Payload Too Large. Can be mitigated by splitting payload in chunks and performing multiple bulk requests.
  2. Large number of threats in alert also overloads browser when alerts loaded and then rendered on alert preview   https://user-images.githubusercontent.com/92328789/205327459-ad3aed57-57ff-42e3-a995-d092fa115e6a.mov

Rendering 1k threats takes significant time and is difficult to navigate. Larger number of threats could even lead to an unresponsive browser's page and eventual crash. One of possibilities to mitigate it is: to limit number of threats saved in alert and/or use virtual scroll rendering on alert page overview

Summary

  1. Limit number of threats loaded in memory (10,000 ?). Downside in this approach, some threats can be missed. But at least it won't lead to Kibana crash
  2. Handle 413 Payload Too Large
  3. Limit number of threat enrichments for alert (1,000 ?), consider virtual scroll rendering for threats in alert overview.

cc: @marshallmain 

nkhristinin commented 1 year ago

@vitaliidm Thanks for this investigation, I have a couple of notes.

getAllThreatListHits used in one place like:

getAllThreatListHits({
      esClient: services.scopedClusterClient.asCurrentUser,
      threatFilters: [...threatFilters, threatFilter],
      query: threatQuery,
      language: threatLanguage,
      index: threatIndex,
      ruleExecutionLogger,
      threatListConfig: {
        _source: [`${threatIndicatorPath}.*`, 'threat.feed.*'],
        fields: undefined,
      },
      pitId: threatPitId,
      reassignPitId: reassignThreatPitId,
      runtimeMappings,
      listClient,
      exceptionFilter,
    });

threatFilter - is a filter generated from potential alerts. So for example we have 2 potential signals with host.name: "host 1" and host.name: "host 2"

Then we try to fetch only threat indicators which have threat.indicator.host.name: "host 1" or threat.indicator.host.name: "host 2"

We also don't control the client data of threat indicators and source events, and the cases which you describe are possible.

Bun in theory it should be really unlikeable that 1 alert has 1000 indicators.

But I also want to check why the preview failed, maybe it's not normal rule execution.

And 413 Payload Too Large - probably should be handled somehow, because the client can have custom max signals. And maybe alerts have not 1000, but 10 threat indicators, anyway after some maxSignals it will cause this error.

vitaliidm commented 1 year ago

@nkhristinin

We also don't control the client data of threat indicators and source events, and the cases which you describe are possible. Bun in theory it should be really unlikeable that 1 alert has 1000 indicators.

Yes, this test scenario is a very extreme case: basically as a wildcard query. Every document will match against every threat. Although unlikely, we need to cover it. As this kind of threat indicator rule easily crashes Kibana and only restart can recover it.

Some kind of limitation of how many indicators we load might do a job. Although, due to the nature of a rule, the size of loaded threat index can vary depends on how many events exists. It's because of matched_queries field in every returned threat. So, if there is 1k events, matched named queries array will consist of 1k items, which is significantly increases size of loaded in memory threats. So, limiting number of loaded threats is not very flexible in that regard, as feasible for Kibana size of threats would depend on number of matched queries. But still, it would prevent Kibana crash.

Here is an example, how response looks when there are too many matched named queries

{
  "pit_id": "08fsAwENdGVzdC1maWxlYmVhdBZzcTVIcEFpeVRPNnlBY1h3ZkUzSF9BABZTV2EzbEtpMVJkT19qaXNfdFpkcHh3AAAAAAAAAA_hFmJBX3lta0NoUlJtNEtZMGh5X3R4aEEAARZzcTVIcEFpeVRPNnlBY1h3ZkUzSF9BAAA=",
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "max_score": null,
    "hits": [
      {
        "_index": "test-filebeat",
        "_id": "QIh_zYQBqSUwB_g_1Djb",
        "_score": null,
        "_source": {
          "threat": {
            "indicator": {
              "geo": {},
              "first_seen": "2022-12-01T11:32:20.000Z",
              "file": {
                "extension": "exe",
                "size": 790528,
                "mime_type": "application/x-dosexec",
                "pe": {
                  "imphash": "f34d5f2d4577ed6d9ceec516c1f5a744"
                },
                "name": "SecuriteInfo.com.Win32.PWSX-gen.8878.21189",
                "hash": {
                  "sha1": "054d5fec9212ea53cb04c1d28a6063c5a4c065c3",
                  "sha384": "70979d04c65b5143f21619063ea1f119c1e868c3cd0c157433d82e8550477113867fa772534e953a18ecea91f561642f",
                  "sha256": "4556daea929e88c1831b8a92814ae2f57b9b8a57be14487a03650ee81d36b67e",
                  "tlsh": "T1BDF4232473F841ACD26F5B7C94B8EA9112BA714B7512F58BDF4430EA2C32F9858136E7",
                  "ssdeep": "24576:kQmnzQE9U3G6PDIaRBNlwCbvk74FRXBaf8i8HtQCZt9lM:kQmzQvGeDrrNzAURw89/O",
                  "md5": "225616a6672687c30a85b7b18467518b"
                },
                "elf": {}
              },
              "provider": "SecuriteInfoCom",
              "type": "file"
            },
            "feed": {
              "name": "[Filebeat] AbuseCH MalwareBazaar",
              "dashboard_id": "ad9c7430-72de-11eb-a3e3-b3cc7c78a70f"
            }
          }
        },
        "sort": [
          0,
          1669894802336,
          0
        ],
        "matched_queries": [
          "yYhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qYhd44QBqSUwB_g_N4_f__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7ohj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qohd44QBqSUwB_g_N4_f__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "44hj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0Yhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pohc44QBqSUwB_g_m4-e__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "u4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "yohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-ohk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tohf44QBqSUwB_g_qI_j__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sIhd44QBqSUwB_g_-48z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5Ihj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0ohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pIhc44QBqSUwB_g_dI-M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-Yhk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "r4hd44QBqSUwB_g_1I8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_4hk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "m4hb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "q4hd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0Ihg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7Ihj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "x4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "A4hk44QBqSUwB_g_sZDx__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uIhf44QBqSUwB_g_z4_r__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2Ihg44QBqSUwB_g_4Y9g__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "Aohk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3Ihi44QBqSUwB_g_GY_i__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uYhf44QBqSUwB_g_9o_7__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "84hk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6Ihj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9Ihk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zYhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pYhc44QBqSUwB_g_dI-M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nIhb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6ohj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "54hj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "n4hb44QBqSUwB_g_Y48h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "z4hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5Yhj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "yIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1ohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "AIhk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rohd44QBqSUwB_g_1I8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "94hk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tIhe44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3Yhi44QBqSUwB_g_QI_u__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "AYhk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9Yhk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zIhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "loha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oohb44QBqSUwB_g_2I9S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "t4hf44QBqSUwB_g_qI_j__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "74hj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "04hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "24hh44QBqSUwB_g_fY-b__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_Ihk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oIhb44QBqSUwB_g_io80__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8Ihj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4ohj44QBqSUwB_g_oI-R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "l4ha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "y4hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-4hk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2ohh44QBqSUwB_g_CI9z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oYhb44QBqSUwB_g_io80__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nohb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_ohk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nYhb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_Yhk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "p4hc44QBqSUwB_g_wo-y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3ohi44QBqSUwB_g_3Y8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4Yhj44QBqSUwB_g_oI-R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "v4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9ohk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "34hj44QBqSUwB_g_BI9D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-Ihk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sohe44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mIha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tYhf44QBqSUwB_g_Wo_B__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1Yhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sYhd44QBqSUwB_g_-48z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "lYha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4Ihj44QBqSUwB_g_BI9D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "s4he44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8ohk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mYha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rIhd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "14hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "w4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "o4hb44QBqSUwB_g__49l__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mohb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2Yhh44QBqSUwB_g_CI9z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5ohj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qIhc44QBqSUwB_g_wo-y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "64hj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rYhd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1Ihg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name"
        ]
      },
      {
        "_index": "test-filebeat",
        "_id": "MogOzYQBqSUwB_g_DjJ9",
        "_score": null,
        "_source": {
          "threat": {
            "indicator": {
              "first_seen": "2022-12-01T09:30:41.000Z",
              "file": {
                "size": 169,
                "pe": {},
                "type": "txt",
                "hash": {
                  "sha256": "2edfb6fa8426ceb68471078d58226a1d5a909da177dfe353abc83f36aa1b9f0f",
                  "tlsh": "T1B9C0800E1744AFB986584797D91ADC46FC6600E0194ABA05168E584C4D1D485B7FCD",
                  "ssdeep": "3:SnfM6TYN9AwLXHPtwrWFFdAFEeIAYRmfgUmjYM1CHaeNMeFy1MFIwporFn:efTWmFvGwEhkHw1MFIsoRn",
                  "md5": "eb1d2306d3f3b347a89782c4cd1eaf18"
                }
              },
              "type": "file"
            },
            "feed": {
              "name": "[Filebeat] AbuseCH Malware",
              "dashboard_id": "ad9c7430-72de-11eb-a3e3-b3cc7c78a70f"
            }
          }
        },
        "sort": [
          1,
          1669887092207,
          1
        ],
        "matched_queries": [
          "yYhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qYhd44QBqSUwB_g_N4_f__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7ohj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qohd44QBqSUwB_g_N4_f__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "44hj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0Yhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pohc44QBqSUwB_g_m4-e__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "u4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "yohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-ohk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tohf44QBqSUwB_g_qI_j__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sIhd44QBqSUwB_g_-48z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5Ihj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0ohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pIhc44QBqSUwB_g_dI-M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-Yhk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "r4hd44QBqSUwB_g_1I8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_4hk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "m4hb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "q4hd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0Ihg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7Ihj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "x4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "A4hk44QBqSUwB_g_sZDx__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uIhf44QBqSUwB_g_z4_r__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2Ihg44QBqSUwB_g_4Y9g__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "Aohk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3Ihi44QBqSUwB_g_GY_i__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uYhf44QBqSUwB_g_9o_7__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "84hk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6Ihj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9Ihk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zYhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pYhc44QBqSUwB_g_dI-M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nIhb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6ohj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "54hj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "n4hb44QBqSUwB_g_Y48h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "z4hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5Yhj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "yIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1ohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "AIhk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rohd44QBqSUwB_g_1I8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "94hk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tIhe44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3Yhi44QBqSUwB_g_QI_u__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "AYhk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9Yhk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zIhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "loha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oohb44QBqSUwB_g_2I9S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "t4hf44QBqSUwB_g_qI_j__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "74hj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "04hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "24hh44QBqSUwB_g_fY-b__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_Ihk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oIhb44QBqSUwB_g_io80__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8Ihj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4ohj44QBqSUwB_g_oI-R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "l4ha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "y4hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-4hk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2ohh44QBqSUwB_g_CI9z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oYhb44QBqSUwB_g_io80__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nohb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_ohk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nYhb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_Yhk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "p4hc44QBqSUwB_g_wo-y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3ohi44QBqSUwB_g_3Y8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4Yhj44QBqSUwB_g_oI-R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "v4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9ohk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "34hj44QBqSUwB_g_BI9D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-Ihk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sohe44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mIha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tYhf44QBqSUwB_g_Wo_B__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1Yhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sYhd44QBqSUwB_g_-48z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "lYha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4Ihj44QBqSUwB_g_BI9D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "s4he44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8ohk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mYha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rIhd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "14hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "w4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "o4hb44QBqSUwB_g__49l__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mohb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2Yhh44QBqSUwB_g_CI9z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5ohj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qIhc44QBqSUwB_g_wo-y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "64hj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rYhd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1Ihg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name"
        ]
      },
      {
        "_index": "test-filebeat",
        "_id": "OYgPzYQBqSUwB_g_VTI4",
        "_score": null,
        "_source": {
          "threat": {
            "indicator": {
              "first_seen": "2022-12-01T09:28:06.000Z",
              "file": {
                "size": 8400040,
                "pe": {
                  "imphash": "ca35138d1be843d89f3a84603af82b86"
                },
                "type": "exe",
                "hash": {
                  "sha256": "85e20ebe81728a940a779fa5930cd4191b02492db6a6e83b4e7f305238958c47",
                  "tlsh": "T1B586C0E1360BE7DFC19504BAD002CE47953983E78701E60BE858B43DBA93E95768FB",
                  "ssdeep": "196608:SxrUIOFnErkFkJ87RpeMBTXAybuMgBPbB:SyIO5ykqJ87rFyy+d",
                  "md5": "0ed0a90f853afe0df37376b6f2898ac1"
                }
              },
              "type": "file"
            },
            "feed": {
              "name": "[Filebeat] AbuseCH Malware",
              "dashboard_id": "ad9c7430-72de-11eb-a3e3-b3cc7c78a70f"
            }
          }
        },
        "sort": [
          2,
          1669887092207,
          2
        ],
        "matched_queries": [
          "yYhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qYhd44QBqSUwB_g_N4_f__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7ohj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qohd44QBqSUwB_g_N4_f__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "44hj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0Yhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pohc44QBqSUwB_g_m4-e__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "u4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "yohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-ohk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tohf44QBqSUwB_g_qI_j__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sIhd44QBqSUwB_g_-48z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5Ihj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0ohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pIhc44QBqSUwB_g_dI-M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-Yhk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "r4hd44QBqSUwB_g_1I8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_4hk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "m4hb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "q4hd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "0Ihg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "7Ihj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "x4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "A4hk44QBqSUwB_g_sZDx__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uIhf44QBqSUwB_g_z4_r__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2Ihg44QBqSUwB_g_4Y9g__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "Aohk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3Ihi44QBqSUwB_g_GY_i__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uYhf44QBqSUwB_g_9o_7__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "84hk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6Ihj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9Ihk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zYhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "pYhc44QBqSUwB_g_dI-M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nIhb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6ohj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "vohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "54hj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "n4hb44QBqSUwB_g_Y48h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "z4hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5Yhj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "yIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1ohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "AIhk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rohd44QBqSUwB_g_1I8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "94hk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tIhe44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3Yhi44QBqSUwB_g_QI_u__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "AYhk44QBqSUwB_g_Y5DM__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9Yhk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zIhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "loha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oohb44QBqSUwB_g_2I9S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "t4hf44QBqSUwB_g_qI_j__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "74hj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "04hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "24hh44QBqSUwB_g_fY-b__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_Ihk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oIhb44QBqSUwB_g_io80__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8Ihj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4ohj44QBqSUwB_g_oI-R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "l4ha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "y4hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-4hk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "uohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2ohh44QBqSUwB_g_CI9z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "oYhb44QBqSUwB_g_io80__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nohb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_ohk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "nYhb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "_Yhk44QBqSUwB_g_Y4_M__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "p4hc44QBqSUwB_g_wo-y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "3ohi44QBqSUwB_g_3Y8s__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4Yhj44QBqSUwB_g_oI-R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "v4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "9ohk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "34hj44QBqSUwB_g_BI9D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "-Ihk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sohe44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mIha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "tYhf44QBqSUwB_g_Wo_B__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1Yhg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "sYhd44QBqSUwB_g_-48z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "lYha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "4Ihj44QBqSUwB_g_BI9D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "s4he44QBqSUwB_g_vo-C__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "8ohk44QBqSUwB_g_PI_A__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mYha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wIhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rIhd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "14hg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "zohg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "w4hg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "o4hb44QBqSUwB_g__49l__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "mohb44QBqSUwB_g_FY8D__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "2Yhh44QBqSUwB_g_CI9z__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "5ohj44QBqSUwB_g_x4-S__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "qIhc44QBqSUwB_g_wo-y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "64hj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "wYhg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "xohg44QBqSUwB_g_RY8h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "rYhd44QBqSUwB_g_Xo_y__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "6Yhj44QBqSUwB_g_x4-T__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name",
          "1Ihg44QBqSUwB_g_uo9R__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name"
        ]
      }
    ]
  }
}

On the other hand, it might worth to invest time into solution that won’t load the whole index into memory, as we basically can't control its size. Or maybe more efficient use of matched queries. As it seems they duplicated for each threat. Also, values in queries are not deduplicated Here is a part of query, that fetches these threats from above. Query value is the same for each filter entry, and such queries can be large number depends on found documents

     {
                        "bool":{
                           "should":[
                              {
                                 "bool":{
                                    "filter":[
                                       {
                                          "bool":{
                                             "should":[
                                                {
                                                   "match":{
                                                      "agent.name":{
                                                         "query":"MacBook-Pro",
                                                         "_name":"mIha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name"
                                                      }
                                                   }
                                                }
                                             ],
                                             "minimum_should_match":1
                                          }
                                       }
                                    ]
                                 }
                              }
                           ],
                           "minimum_should_match":1
                        }
                     },
                     {
                        "bool":{
                           "should":[
                              {
                                 "bool":{
                                    "filter":[
                                       {
                                          "bool":{
                                             "should":[
                                                {
                                                   "match":{
                                                      "agent.name":{
                                                         "query":"MacBook-Pro",
                                                         "_name":"mYha44QBqSUwB_g_7Y_x__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name"
                                                      }
                                                   }
                                                }
                                             ],
                                             "minimum_should_match":1
                                          }
                                       }
                                    ]
                                 }
                              }
                           ],
                           "minimum_should_match":1
                        }
                     },
                     {
                        "bool":{
                           "should":[
                              {
                                 "bool":{
                                    "filter":[
                                       {
                                          "bool":{
                                             "should":[
                                                {
                                                   "match":{
                                                      "agent.name":{
                                                         "query":"MacBook-Pro",
                                                         "_name":"lYha44QBqSUwB_g_xo_h__SEP__.ds-auditbeat-8.4.2-2022.11.28-000001__SEP__host.name__SEP__agent.name"
                                                      }
                                                   }
                                                }
                                             ],
                                             "minimum_should_match":1
                                          }
                                       }
                                    ]
                                 }
                              }
                           ],
                           "minimum_should_match":1
                        }
                     },

But I also want to check why the preview failed, maybe it's not normal rule execution.

It failed due too Kibana out of memory as in example before

Here is also an example of problematic rule:




{"id":"167fef80-748c-11ed-917a-6de2bc6f4bd5","updated_at":"2022-12-05T16:53:50.194Z","updated_by":"elastic","created_at":"2022-12-05T11:00:54.012Z","created_by":"elastic","name":"threat indicator [Duplicate] [Duplicate]","tags":[],"interval":"61m","enabled":false,"description":"threat indicator","risk_score":21,"severity":"low","license":"","output_index":"","meta":{"from":"5m","kibana_siem_app_url":"http://localhost:5603/kbn/app/security"},"author":[],"false_positives":[],"from":"now-3960s","rule_id":"d672511e-497e-453c-8b51-ca7898dc723f","max_signals":100,"risk_score_mapping":[],"severity_mapping":[],"threat":[],"to":"now","references":[],"version":7,"exceptions_list":[],"immutable":false,"related_integrations":[],"required_fields":[],"setup":"","type":"threat_match","language":"kuery","index":["apm-*-transaction*","auditbeat-*","endgame-*","filebeat-*","logs-*","packetbeat-*","traces-apm*","winlogbeat-*","-*elastic-cloud-logs-*"],"query":"*:*","filters":[],"threat_filters":[],"threat_query":"@timestamp >= \"now-30d/d\"","threat_mapping":[{"entries":[{"field":"host.name","type":"mapping","value":"agent.name"}]}],"threat_language":"kuery","threat_index":["logs-ti_*","filebeat-*"],"threat_indicator_path":"threat.indicator","throttle":"no_actions","actions":[]}
{"exported_count":1,"exported_rules_count":1,"missing_rules":[],"missing_rules_count":0,"exported_exception_list_count":0,"exported_exception_list_item_count":0,"missing_exception_list_item_count":0,"missing_exception_list_items":[],"missing_exception_lists":[],"missing_exception_lists_count":0}

And 413 Payload Too Large - probably should be handled somehow, because the client can have custom max signals. And maybe alerts have not 1000, but 10 threat indicators, anyway after some maxSignals it will cause this error.

In this particular case, it caused by large number of threats. More than 1k threats per one alert, with batch of 100 alerts exceeds payload limit.

vitaliidm commented 1 year ago

Initial work was done in https://github.com/elastic/kibana/pull/147454, that prevents crash during single rule run.

Continue work on IM rule in https://github.com/elastic/kibana/issues/148821