Fix ES|QL siem_rule_ndjson Detection Rule Import into Kibana

[Mat0vu]

Hi everyone,

after translating a sigma rule into siem_rule_ndjson format using the target esql, I encountered an error when importing this rule into Kibanas Detection Engine:

Short Error: "investigation_fields.field_names: Required" Full Error:

{
  "name": "Network errors",
  "raw_network_error": {
    "exceptions_success": true,
    "exceptions_success_count": 0,
    "exceptions_errors": [],
    "rules_count": 1,
    "success": false,
    "success_count": 0,
    "errors": [
      {
        "rule_id": "(unknown id)",
        "error": {
          "status_code": 400,
          "message": "investigation_fields.field_names: Required"
        }
      }
    ],
    "action_connectors_errors": [],
    "action_connectors_success": true,
    "action_connectors_success_count": 0
  },
  "message": "investigation_fields.field_names: Required"
}

This issue only seems to appear when using the siem_rule_ndjson format and the target esql. The backend always creates an empty object { "investigation_fields": {} } because in the docs it says that investigation_fields needs to be an object. This however causes the error shown above.

There are various options to fix this issue:

• Convert investigation_fields from an empty object to an empty list -> this does not cause an exception when importing, but violates the API docs stating that investigation_fields has to be an object
• Populate the object (at least one field is required) -> e.g. { "investigation_fields": {"field_names": ["destination.ip"]} }

Both of these options are not good in my opinion as they only serve as a hacky workaround to make the import work. Since the code does not fill the key investigation_fields with any content at the moment and it is stated to be an optional field anyways, I would suggest to remove the key from the template completely. Regular rules created manually using the GUI do not contain the key either when exporting them (I tried out some options and the export never contained this attribute). If somebody really needs this attribute, I think it could probably be added again via a pipeline.

What do you think?

[andurin]

Thank you. Merged.