Added new alerter to send alerts to Opensearch

[OlehPalanskyi]

Description

Hi. Problem: I have a needs to send alerts to the opensearch index.

Solution: I have added a new alerter to send alerts to Opensearch

Description: Create and manage separately index for all alerts for statistics and report purpose.

Opensearch alerter can be used to create a new alert in existen Opensearch. The alerter supports custom fields, and observables from the alert matches and rule data.

Required:

opensearch_alert_config: Configuration options for the alert, see example below for structure.

customFields Fields must be manually added, all of them will exist in the newly created index. You can set own field or use existed field fron match(see example below for structure).

index_alerts_name: This field setup the output index for alerts.

One of below is required:

opensearch_connection: Options the connection details to your instance (see example below for the required syntax Example 1).

opensearch_config: Options for the get connection details to your instance from file (see example below for the required syntax Example 2).

Example 1 usage::

alert: opensearch

opensearch_connection:
  es_host: localhost
  es_port: es_port
  ssl_show_warn: False
  use_ssl: True
  verify_certs: False
  es_username: user
  es_password: password
  index_alerts_name: opensearch_elastalert2               # You can create own config or use global config just added ``index_alerts_name`` in global config

opensearch_alert_config:
  #Existing fields from match alert
  message: message
  host.name: host.name
  event.action: event.action
  event.type: event.type
  winlog.computer_name: winlog.computer_name
  winlog.event_id: winlog.event_id
  winlog.task: winlog.task
  #Enrich existen event with additional fields
  customFields:
    - name: original_time
      value: "@timestamp"
    - name: severity
      value: high
    - name: risk_score
      value: 73
    - name: description
      value: General description.

Example 2 usage::

alert: opensearch

opensearch_config: /opt/elastalert/config/config.yaml       # You can create own config or use global config just added ``index_alerts_name`` in global config

opensearch_alert_config:
  #Existing fields from match alert
  message: message
  host.name: host.name
  event.action: event.action
  event.type: event.type
  winlog.computer_name: winlog.computer_name
  winlog.event_id: winlog.event_id
  winlog.task: winlog.task
  #Enrich existen event with additional fields
  customFields:
    - name: original_time
      value: "@timestamp"
    - name: severity
      value: high
    - name: risk_score
      value: 73
    - name: description
      value: General description.

Checklist

• [x] I have reviewed the contributing guidelines.
• [x] I have included unit tests for my changes or additions.
• [x] I have successfully run make test-docker with my changes.
• [x] I have manually tested all relevant modes of the change in this PR.
• [x] I have updated the documentation.
• [x] I have updated the changelog.

Questions or Comments

unit tests

[jertel]

Hello, this is an interesting alerter. It looks like it could be made to work with both Elasticsearch and OpenSearch instead of just OpenSearch, since it's already using the Elasticsearch Python SDK. Perhaps it (and its settings) could be renamed to index or indexer, something along those lines and make it agnostic to either OpenSearch or Elasticsearch.

Aside from that discussion, the missing unit tests are important so those should be included. Since you've done a good job breaking the logic into discrete functions it should be relatively light work to add that coverage. We aim for 100% coverage on all new code. A side effect of doing so is it lets the PR reviewers better understand what the inputs and outputs look like for each function.

Also, a couple instances of existen and existed typos should be corrected to existing in the docs.

Thanks for taking the time to submit this PR.

[OlehPalanskyi]

Thank you for your feedback. I'll updated my patch based on your suggestions.

[OlehPalanskyi]

Hello, I updated my patch based on your suggestions.

Indexer
Description: Create and manage separately index for all alerts for statistics and report purpose.

Indexer alerter can be used to create a new alert in existing Opensearch/Elasticsearch. The alerter supports
custom fields, and observables from the alert matches and rule data.

Required:

``indexer_alert_config``: Configuration options for the alert, see example below for structure.

``customFields`` Fields must be manually added, all of them will exist in the newly created index. You can set own field or use existing field fron match(see example below for structure).

``index_alerts_name``: This field setup the output index for alerts.

One of below is required:

``indexer_connection``: Options the connection details to your instance (see example below for the required syntax Example 1).

``indexer_config``: Options for the get connection details to your instance  from file (see example below for the required syntax Example 2).

Example 1 usage::

    alert: indexeralerter

    indexer_connection:
      es_host: localhost
      es_port: es_port
      ssl_show_warn: False
      use_ssl: True
      verify_certs: False
      es_username: user
      es_password: password
      index_alerts_name: elastalert2               # You can create own config or use global config just added ``index_alerts_name`` in global config

    indexer_alert_config:
      #Existing fields from match alert
      message: message
      host.name: host.name
      event.action: event.action
      event.type: event.type
      winlog.computer_name: winlog.computer_name
      winlog.event_id: winlog.event_id
      winlog.task: winlog.task
      #Enrich existing event with additional fields
      customFields:
        - name: original_time
          value: "@timestamp"
        - name: severity
          value: high
        - name: risk_score
          value: 73
        - name: description
          value: General description.

Example 2 usage::

    alert: indexeralerter

    indexer_config: /opt/elastalert/config/config.yaml       # You can create own config or use global config just added ``index_alerts_name`` in global config

    indexer_alert_config:
      #Existing fields from match alert
      message: message
      host.name: host.name
      event.action: event.action
      event.type: event.type
      winlog.computer_name: winlog.computer_name
      winlog.event_id: winlog.event_id
      winlog.task: winlog.task
      #Enrich existing event with additional fields
      customFields:
        - name: original_time
          value: "@timestamp"
        - name: severity
          value: high
        - name: risk_score
          value: 73
        - name: description
          value: General description.

[OlehPalanskyi]

[jertel]

Nice job!

Will you please rename the alerter to "indexer" since none of the other alerters have that suffix?

Ex:

alert: indexer

The class name of IndexerAlerter is fine, so it just the lookup name that needs changed.

[jertel]

Thanks. Two remaining items:

1. Update the CHANGELOG.md file. You can add a new line under the New Features section of 2.TBD.TBD. Follow the same patterns from the previous releases.
2. Update the elastalert/schema.yaml, which helps ElastAlert 2 know that the user has supplied the required and optional parameters for this new alerter.

[jertel]

Also, I noticed that there is an inconsistently named property: index_alerts_name. All alerter specific props should consistently begin with the alerter name as the prefix. Ex: indexer_alerts_index might be more suitable here.

[OlehPalanskyi]

I corrected my little mistakes. In the future, I will not make such mistakes. Thank you.

[jertel]

Looks good. I'll leave this PR open for a few days, for additional comments by the community.

[OlehPalanskyi]

Thank you.