search

uken / fluent-plugin-elasticsearch

Apache License 2.0
890 stars 310 forks source link

UnsupportedProductError with AWS ES 7.10 #952

Closed f0o closed 2 years ago

f0o commented 2 years ago

(check apply)

Problem

Using version 5.2 of this plugin it seems not to be possible to publish logs into AWS ES 7.10

The error I get is:

2022-02-25 10:14:33 +0000 [error]: #0 unexpected error error_class=Elasticsearch::UnsupportedProductError error="The client noticed that the server is not a supported distribution of Elasticsearch."

I've already tried to pin the elasticsearch gem's version as mentioned in https://github.com/uken/fluent-plugin-elasticsearch/issues/951#issuecomment-1050595741 with no effect.

Steps to replicate

Literally just try to use AWS ES 7.10 with the latest plugin version and watch it fail.

Expected Behavior or What you need to ask

Publishing logs into ES

Using Fluentd and ES plugin versions

f0o commented 2 years ago

It's probably worth mentioning that using:

verify_es_version_at_startup false
default_elasticsearch_version 7

did not help either. (Other than not getting an error in the stderr anymore, but still no data being pushed to ES)

In case you need it here's what AW ES 7.10 reports as versions:

{
  "name" : "redacted",
  "cluster_name" : "redacted",
  "cluster_uuid" : "redacted",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "unknown",
    "build_date" : "2021-09-29T11:42:59.634166Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
f0o commented 2 years ago

I got it to work again by specifically installing version 7.10 from elasticsearch gem. <14 did not work only fluent-gem install elasticsearch --version '= 7.10' did.