[ ] (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.
Problem
elasticsearch_data_stream mandates host to be configured even though hosts is configured. Otherwise following error is seen in the logs.
failed to flush the buffer. retry_time=0 next_retry_seconds=2023-02-07 21:08:54 +0000 chunk="5f41e9b7cda5b02d1982342447579" error_class=Fluent ::ConfigError error="Failed to create data stream: <datastream-test-application-test> Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED)"
This issue might seem familiar because the same problem was posted by someone else while using the opensearch plugin. The problem here is that the documentation doesn't clearly say that the use of "hosts" is not compatible with the use of data_streams
That can be found here https://github.com/fluent/fluent-plugin-opensearch/issues/5
...
Result of the fluent-gem list:
fluent-plugin-elasticsearch (5.2.4, 5.1.5)
The version 5.1.5 apparently comes shipped with the fluentd-elastic image of the chart and can't be remove (or I don't know how to do it if it can). The version 5.2.4 is installed by using the helm value plugins of the helm chart. According to fluentd documentation, if there are multiple versions then the most recent version is the one loaded by fluentd. Which mean in this case the version 5.2.4 (Correct me if I'm wrong).
Problem
elasticsearch_data_stream mandates host to be configured even though hosts is configured. Otherwise following error is seen in the logs.
failed to flush the buffer. retry_time=0 next_retry_seconds=2023-02-07 21:08:54 +0000 chunk="5f41e9b7cda5b02d1982342447579" error_class=Fluent ::ConfigError error="Failed to create data stream: <datastream-test-application-test> Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED)"This issue might seem familiar because the same problem was posted by someone else while using the opensearch plugin. The problem here is that the documentation doesn't clearly say that the use of "hosts" is not compatible with the use of data_streams That can be found here https://github.com/fluent/fluent-plugin-opensearch/issues/5 ...
Steps to replicate
Expected Behavior or What you need to ask
As the documentation says: If you specify hosts option, host and port options are ignored. https://github.com/uken/fluent-plugin-elasticsearch#hosts
Using Fluentd and ES plugin versions
The fluend helm chart version 0.3.9
Result of the fluent-gem list:
fluent-plugin-elasticsearch (5.2.4, 5.1.5)The version 5.1.5 apparently comes shipped with the fluentd-elastic image of the chart and can't be remove (or I don't know how to do it if it can). The version 5.2.4 is installed by using the helm value plugins of the helm chart. According to fluentd documentation, if there are multiple versions then the most recent version is the one loaded by fluentd. Which mean in this case the version 5.2.4 (Correct me if I'm wrong).