Docker-compose example does not work correctly

ozang commented 3 years ago

Hi, docker-compose example in page https://docs.fluentd.org/container-deployment/docker-compose does not work correctly. Kibana shows only shows errors saying "there is a tag error." I cannot see any access log related web access.

kenhys commented 3 years ago

Hmm, it seems that it just works for me. Does httpd image up and running correctly?

ozang commented 3 years ago

Actually the latest image in the page https://docs.fluentd.org/container-deployment/docker-compose shows the error's reason. I mean shared picture already says the reason, It seems to be a config problem.

sebkraemer commented 2 years ago

I'm too was trying to get that example running. There several things off between the tutorial and the actual result.

The web service won't return <html><body><h1>It works!</h1></body></html> without any configuration, and none is provided.
The tag httpd.access won't appear in kibana, that's something I was missing and led me to this issue.
fluent.conf in docs and in this repo differ, see below.

From my experiments, I can say that:

FROM fluent/fluentd:v1.14-1

USER root

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.1.4"]

updating fluentd and elasticsearch

and -- maybe more importantly -- omitting the <label @FLUENT_LOG> part from fluent.conf fixed the issue for me. Note that the conf file change is a key difference between the tutorial code and code in this repo. This also was the key difference between a working and nonworking solution for me.

With it, as @ozang reported, I'm getting the tag error, and this is the only tagged message in ELK later on. It appears that with it, fluentd won't recognize the "match *.**" clause as mathing for that tag.

I have minor docker-compose.yml changes locally because of my project but I think these are not relevant. @ozang, maybe you want to try with a current fluentd and leaving out the label in the fluentd configuration.

If there is interest, I volunteer to rewalk the example and create a PR. But please consider, I'm absolutely new to fluentd so I don't know 100% what I'll be doing.

daipom commented 1 year ago

Actually the latest image in the page https://docs.fluentd.org/container-deployment/docker-compose shows the error's reason. I mean shared picture already says the reason, It seems to be a config problem.

Actually, the image of that page says no patterns matched...

https://github.com/fluent/fluentd-docs-gitbook/blob/1.0/.gitbook/assets/7.10_efk-kibana-discover.png

I don't know the situation, but there is no problem with the Fluentd config on that page, right?

and -- maybe more importantly -- omitting the <label @FLUENT_LOG> part from fluent.conf fixed the issue for me.

This is for the config on the following repository, right?

https://github.com/digikin/fluentd-elastic-kibana

There are other reports of the information on the page not working. We need to fix it.

If there is interest, I volunteer to rewalk the example and create a PR. But please consider, I'm absolutely new to fluentd so I don't know 100% what I'll be doing.

Sorry for the delay. If it is still possible, I would appreciate it if you could try to make a PR.

Note that we need to update the 1.0 branch. We will review it, so no worries!

sebkraemer commented 10 months ago

Sorry for the delay. If it is still possible, I would appreciate it if you could try to make a PR.

Hi @daipom, sorry for getting back to you so late and with bad news. But I used the setup only for a small POC during a learning session and I have that setup or could get it up (and fixed properly) easily. Haven't used fluentd since..

fluent / fluentd-docs-gitbook

Docker-compose example does not work correctly #297