Open stemabo opened 2 years ago
Can confirm as I'm having the exact same issue. Been having a bunch of trouble trying to get the EFK stack working. Seems like a lot of articles and tutorials out there are out of date. A real shame that even the official documentation doesn't work.
Hi!
I came across the same Problem and I got it to work. Not sure if this solution fits for your situation.
While following the official docs ( https://docs.fluentd.org/container-deployment/docker-compose ) I changed the config to use the newest versions up to date.
1) In the fluentd Dockerfile change the version from "5.0.3" to the newest version
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]
2) Since this only seems to work with elastic >= 8, you have to update elastic and kibana in the docker-compose.yml as well. I also needed to disable TLS for elastic as well.
version: "3"
services:
[...]
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.1.2
container_name: elasticsearch
environment:
- "discovery.type=single-node"
- xpack.security.enabled=false
expose:
- "9200"
ports:
- "9200:9200"
kibana:
image: docker.elastic.co/kibana/kibana:8.1.2
[...]
This works for me. Of course, disabling security (xpack.security.enabled=false) is just for proof of concept. Otherwise elastic will complain about the http connections where it expects https. Right now I am not aware how to configure fluentd to do this.
Hope this helps! Cheers
Thank you @Xernosch I have suffered for 1 week beacuse of version compatibility of fluentd ... It works perfectely!!
- In the fluentd Dockerfile change the version from "5.0.3" to the newest version
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]
- Since this only seems to work with elastic >= 8, you have to update elastic and kibana in the docker-compose.yml as well. I also needed to disable TLS for elastic as well.
If you don't use ES8 for any reasons install elasticsearch gem version < 8 just before fluent-plugin-elasticsearch:
RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "< 8"]
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]
and don't switch to ES8 in docker-compose.yml
Hi, I tried to apply the suggested approach. Now I receive a new error:
2023-07-19 15:24:59 +0000 [error]: #0 unexpected error error_class=Errno::EACCES error="Permission denied @ dir_s_mkdir - /var/log/td-agent"
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:247:in `mkdir'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:247:in `fu_mkdir'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:228:in `block (2 levels) in mkdir_p'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:226:in `reverse_each'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:226:in `block in mkdir_p'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:211:in `each'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:211:in `mkdir_p'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/plugin/in_tail.rb:243:in `start'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:203:in `block in start'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:191:in `each'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:191:in `block in lifecycle'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:178:in `each'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:178:in `lifecycle'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:202:in `start'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/engine.rb:248:in `start'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/engine.rb:147:in `run'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:720:in `block in run_worker'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:971:in `main_process'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:711:in `run_worker'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/command/fluentd.rb:376:in `<top (required)>'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/bin/fluentd:15:in `<top (required)>'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:23:in `load'
2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:23:in `<main>'
2023-07-19 15:24:59 +0000 [error]: #0 unexpected error error_class=Errno::EACCES error="Permission denied @ dir_s_mkdir - /var/log/td-agent"
2023-07-19 15:24:59 +0000 [error]: #0 suppressed same stacktrace
2023-07-19 15:24:59 +0000 [error]: Worker 0 finished unexpectedly with status 1
elasticsearch/kibana version: 7.10.2 fluentd version: v1.12.0-debian-1.0
Any suggestion on why having this error and a possible solution?
Thank you in advance!
For those having problems, it all seems to come down to version incompatibility. This worked for me:
version: "3"
services:
web:
container_name: web
image: httpd
ports:
- "80:80"
links:
- fluentd
logging:
driver: "fluentd"
options:
fluentd-address: localhost:24224
tag: httpd.access
fluentd:
container_name: fluentd
build: ./fluentd
volumes:
- ./fluentd/conf:/fluentd/etc
links:
- "elasticsearch"
ports:
- "24224:24224"
- "24224:24224/udp"
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.1.2
container_name: elasticsearch
environment:
- "discovery.type=single-node"
- "xpack.security.enabled=false"
expose:
- "9200"
ports:
- "9200:9200"
kibana:
container_name: kibana
image: docker.elastic.co/kibana/kibana:8.1.2
links:
- "elasticsearch"
ports:
- "5601:5601"
With the Dockerfile being
# fluentd/Dockerfile
FROM fluent/fluentd:v1.12.0-debian-1.0
USER root
RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "8.1.2"]
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]
USER fluent
You might need to restart the fluentd pod if it fails before the elasticsearch pod has finished initialising
mark
elasticsearch 8.x dockerfile
FROM fluent/fluentd:v1.16.2-debian-1.1
USER root
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.3.0"]
USER fluent
docker-compose.yaml
version: "3"
services:
web:
image: httpd
ports:
- "80:80"
links:
- fluentd
logging:
driver: "fluentd"
options:
fluentd-address: localhost:24224
tag: httpd.access
fluentd:
image: registry.cn-shenzhen.aliyuncs.com/cnregistry/fluentd-es:v1.16.2-debian-1.1
volumes:
- ./fluentd/conf:/fluentd/etc
links:
- "elasticsearch"
ports:
- "24224:24224"
- "24224:24224/udp"
elasticsearch:
container_name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:8.10.2
ports:
- 9200:9200
environment:
- http.host=0.0.0.0
- transport.host=127.0.0.1
- xpack.security.enabled=false
kibana:
image: docker.elastic.co/kibana/kibana:8.10.2
links:
- "elasticsearch"
ports:
- "5601:5601"
@paulsjohnson91 Thanks for your notes! It works! ❤️ ❤️ ❤️
Describe the bug
I am (to the best of my knowledge) strictly following the official docker-compose example (https://docs.fluentd.org/container-deployment/docker-compose) but the Fluentd image runs into an error installing
fluent-plugin-elasticsearch
:fluentd_1 | 2022-02-21 08:44:31 +0000 [info]: gem 'fluentd' version '1.12.0'
fluentd_1 | /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in 'require': cannot load such file -- elasticsearch/transport/transport/connections/selector (LoadError)
Is also experienced by others (@https://stackoverflow.com/users/18044852/henry-kao): https://stackoverflow.com/questions/71120621/efk-system-is-build-on-docker-but-fluentd-cant-start-up/71126320
To Reproduce
follow https://docs.fluentd.org/container-deployment/docker-compose
Expected behavior
Fluentd container to start up with no errors...
Your Environment
Your Configuration
Your Error Log
Additional context
I would assume this is me getting something wrong, but as others confirm this behavior it might actually be an issue...