opensearch-project / logstash-output-opensearch

A Logstash plugin that sends event data to a OpenSearch clusters and stores as an index.
https://opensearch.org/docs/latest/clients/logstash/index/
Apache License 2.0
106 stars 80 forks source link

Support for ElasticSearch 6.x clusters #123

Open dlvenable opened 2 years ago

dlvenable commented 2 years ago

Is your feature request related to a problem? Please describe.

The logstash-output-amazon_es plugin provides SigV4 signing for AWS OpenSearch Service clusters. It works with Elasticsearch version 6.5 and above. Now that the logstash-output-opensearch plugin supports SigV4 signing, it is very similar to the amazon_es plugin. I believe the logstash-output-opensearch plugin can replace the amazon_es plugin.

One key difference is that the logstash-output-opensearch plugin does not currently support Elasticsearch 6.x clusters.

Describe the solution you'd like

Support Elasticsearch 6.x clusters.

Describe alternatives you've considered

An alternative is to require teams with 6.x clusters to use the amazon_es plugin. But, this is likely to confuse developers since there are two plugins with very similar functionality. Additionally, some teams may have clusters with multiple versions.

dlvenable commented 2 years ago

The amazon_es plugin currently uses at the major version to determine which template to use.

It then loads either elasticsearch-template-es6x.json or elasticsearch-template-es7x.json.

The logstash-output-opensearch plugin would need to be able to detect which version of OpenSearch or Elasticsearch is running. One obvious solution is to use the distribution, but this can cause other problems. So I think this current issue relies on a solution to #66.