jertel / elastalert2

ElastAlert 2 is a continuation of the original yelp/elastalert project. Pull requests are appreciated!
https://elastalert2.readthedocs.org
Apache License 2.0
859 stars 277 forks source link

esversion checks for opensearch number 1 and 7 #1449

Closed 32bitbradley closed 1 month ago

32bitbradley commented 1 month ago

Added support for a reported esversion number that starts with 1 and 7 to determine a cluster is es 7.10.2 like

Description

When trying to get elastalert 2 to work with Wazuh, and thier version of OpenSearch, Wazuh Indexer, I noticed that I was getting the below error:

elastalert-1  | WARNING:elasticsearch:PUT https://es:9200/elastalert_status/_doc/_mapping?include_type_name=true [status:400 request:0.005s]
elastalert-1  | Traceback (most recent call last):
elastalert-1  |   File "/usr/local/bin/elastalert-create-index", line 8, in <module>
elastalert-1  | Reading Elastic 7 index mappings:
elastalert-1  | Reading index mapping 'es_mappings/7/silence.json'
elastalert-1  | Reading index mapping 'es_mappings/7/elastalert_status.json'
elastalert-1  | Reading index mapping 'es_mappings/7/elastalert.json'
elastalert-1  | Reading index mapping 'es_mappings/7/past_elastalert.json'
elastalert-1  | Reading index mapping 'es_mappings/7/elastalert_error.json'
elastalert-1  |     sys.exit(main())
elastalert-1  |              ^^^^^^
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elastalert/create_index.py", line 242, in main
elastalert-1  |     create_index_mappings(es_client=es, ea_index=index, recreate=args.recreate, old_ea_index=old_index)
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elastalert/create_index.py", line 78, in create_index_mappings
elastalert-1  |     es_client.indices.put_mapping(index=ea_index, doc_type='_doc',
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
elastalert-1  |     return func(*args, params=params, headers=headers, **kwargs)
elastalert-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elasticsearch/client/indices.py", line 408, in put_mapping
elastalert-1  |     return self.transport.perform_request(
elastalert-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elasticsearch/transport.py", line 392, in perform_request
elastalert-1  |     raise e
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elasticsearch/transport.py", line 358, in perform_request
elastalert-1  |     status, headers_response, data = connection.perform_request(
elastalert-1  |                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elasticsearch/connection/http_requests.py", line 199, in perform_request
elastalert-1  |     self._raise_error(response.status_code, raw_data)
elastalert-1  |   File "/usr/local/lib/python3.12/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
elastalert-1  |     raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elastalert-1  | elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'request [/elastalert_status/_doc/_mapping] contains unrecognized parameter: [include_type_name]')
elastalert-1 exited with code 1

There is some code that checks if the target elasticSearch/OpenSearch cluster is OpenSearch 1.x (like elasticSearch 7.10.2) or not, and therefore presumed to be like elasticSearch 8.x.

            if esinfo.get('distribution') == "opensearch":
                # https://opensearch.org/
                if esversion[0] == "1":
                    # OpenSearch 1.x is based on Elasticsearch 7.10.2
                    esversion = "7.10.2"
                else:
                    # OpenSearch 2.x has qualities similar to 8.2.0
                    esversion = "8.2.0"

In the wazuh indexer version however the number in the opensearch version response is 7.10.2, and not 1.x as expected. This causes the wrong version of elasticSearch to be assumed resulting in further requests including the unsupported include_type_name parameter, causing the above issue.

Example GET / response from wazuh indexer

{
  "name": "wazuh.indexer",
  "cluster_name": "opensearch",
  "cluster_uuid": "oWZKY5bPRE2__HHlNVMKIg",
  "version": {
    "number": "7.10.2",
    "build_type": "rpm",
    "build_hash": "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
    "build_date": "2023-06-03T06:24:25.112415503Z",
    "build_snapshot": false,
    "lucene_version": "9.6.0",
    "minimum_wire_compatibility_version": "7.10.0",
    "minimum_index_compatibility_version": "7.0.0"
  },
  "tagline": "The OpenSearch Project: https://opensearch.org/"
}

This PR fixes that by changing the check to see if the number field stars with either 1 or 7 to assume a 7.10.2 like cluster.

Checklist

Questions or Comments