uken / fluent-plugin-elasticsearch

Apache License 2.0
890 stars 310 forks source link

cannot connection to elasticsearch with this plugin #974

Closed one-percent-of closed 2 years ago

one-percent-of commented 2 years ago

Problem

cannot connection to elasticsearch with this plugin

Configure elasticsearch server based on the URL(https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html). Configure fluentd server based on the URL(https://docs.fluentd.org/container-deployment/docker-compose).

fluentd docker image version : v1.14-debian-1 elasticsearch docker image version : 8.2.0 this fluent-plugin-elasticsearch plugin version : 5.2.2

Configured as shown in each official document, but no connection!

log

2022-05-20 00:53:19 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf" 2022-05-20 00:53:19 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.2.2' 2022-05-20 00:53:19 +0000 [info]: gem 'fluentd' version '1.14.6' 2022-05-20 00:53:21 +0000 [info]: 'flush_interval' is configured at out side of . 'flush_mode' is set to 'interval' to keep existing behaviour 2022-05-20 00:53:21 +0000 [warn]: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead 2022-05-20 00:53:21 +0000 [info]: using configuration file:

@type forward
port 24224
bind "0.0.0.0"

<match *.**> @type copy

@type "elasticsearch" host "127.0.0.1" port 9200 scheme https ssl_verify false ssl_version TLSv1_2 ca_file "/var/lib/docker/volumes/efk_certs/_data/ca/ca.crt" user "elastic" password xxxxxx logstash_format true logstash_prefix "fluentd" logstash_dateformat "%Y%m%d" include_tag_key true include_timestamp true type_name "access_log" tag_key "@log_name" flush_interval 1s flush_interval 1s
<store>
  @type "stdout"
</store>

2022-05-20 00:53:21 +0000 [info]: starting fluentd-1.14.6 pid=7 ruby="2.7.5" 2022-05-20 00:53:21 +0000 [info]: spawn command to main: cmdline=["/usr/local/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/local/bundle/bin/fluentd", "--config", "/fluentd/etc/fluent.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"] 2022-05-20 00:53:25 +0000 [info]: adding match pattern="*.**" type="copy" 2022-05-20 00:53:27 +0000 [info]: #0 'flush_interval' is configured at out side of . 'flush_mode' is set to 'interval' to keep existing behaviour The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-20 00:53:29 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:53:29 +0000 [warn]: #0 Remaining retry: 14. Retry to communicate after 2 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-20 00:53:33 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:53:33 +0000 [warn]: #0 Remaining retry: 13. Retry to communicate after 4 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-20 00:53:41 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:53:41 +0000 [warn]: #0 Remaining retry: 12. Retry to communicate after 8 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-20 00:53:57 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:53:57 +0000 [warn]: #0 Remaining retry: 11. Retry to communicate after 16 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-20 00:54:29 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:54:29 +0000 [warn]: #0 Remaining retry: 10. Retry to communicate after 32 second(s). 2022-05-20 00:55:33 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:55:33 +0000 [warn]: #0 Remaining retry: 9. Retry to communicate after 64 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-20 00:57:41 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED) 2022-05-20 00:57:41 +0000 [warn]: #0 Remaining retry: 8. Retry to communicate after 128 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product.

one-percent-of commented 2 years ago

Remove the old version of elasticsearch from the fluent.conf file and reinstall the version i want to use for it to work properly.

FROM fluent/fluentd:v1.14-debian-1 USER root RUN gem uninstall -I elasticsearch && gem install elasticsearch -v 8.2.0 RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"] USER fluent