search

willfarrell / docker-filebeat

Docker image for Elastic Filebeat
ISC License
11 stars 9 forks source link

Use tail -f instead of cat #4

Open hamsterready opened 7 years ago

hamsterready commented 7 years ago

In the wrapper script:

https://github.com/willfarrell/docker-filebeat/blob/e1b40b46dc9edca80e37527f26bbf2c8e61917fe/5-stdin/docker-entrypoint.sh#L29

I would use use tail -f instead of cat.

Otherwise on container restart I was experiencing issue similar to this one: https://github.com/elastic/beats/issues/3315

with error message;

Err State for should have been dropped, but couldn't as state is not finished.
willfarrell commented 7 years ago

Thanks! Fixed, let me know how it works out.

hamsterready commented 7 years ago

When monitored containers are restarted/recreated then I see nice:

ubuntu@backend-03:~$ docker logs -f e817f9b3ce90 2>&1 | grep Processing
Processing rta_metrics-consumer.d5fvj98is8xookf2l8534isse.svg2txg6i3jvk35aa5ndsce6o ...
Processing rta_metrics-api.d5fvj98is8xookf2l8534isse.7u0d49rre4agmtxytt7jrnfej ...
Processing rta_metrics-consumer.d5fvj98is8xookf2l8534isse.3ipv9bi3mrnbmnzp6nicxfg8e ...
Processing rta_metrics-consumer.d5fvj98is8xookf2l8534isse.k19hs64jk4xu7nu70oowqph9f ...
Processing rta_metrics-consumer.d5fvj98is8xookf2l8534isse.lnphgm20jknzrvk8f84xl9wee ...
Processing rta_metrics-consumer.d5fvj98is8xookf2l8534isse.xle9xx76md5unczpt08vbx4f5 ...

instead of 

Err State for should have been dropped, but couldn't as state is not finished.

Cheers,

hamsterready commented 7 years ago

hi @willfarrell

I think I was a bit too early with confirming all is good.

The tail -f on fifo works in a bit different way then I was expected https://stackoverflow.com/questions/20409071/tail-f-on-a-named-pipe-behaves-strange and as such when tail is being used one will lose the data.

On the other hand when cat is being used the filebeat will crash/hang on container restart and will stop delivering logs to logstash or whatever is configured.

In essence the filebeat becomes useless in docker environment - lovely.

willfarrell commented 7 years ago

So ... we have to choose the lesser of two evils.

hamsterready commented 7 years ago

Man, to be honest I decided to go with logspout... and then was hit by this issue https://github.com/gliderlabs/logspout/issues/330 bad luck