When I use the fluent/fluentd:v1.1.0 image the in_tail plugin fallbacks to polling file every 5 seconds instead of using inotify. This causes loss in timestamp precision. fluent/fluentd:v1.1.0-debian image handles watching file correctly and does not fallback to polling.
Steps to reproduce
put Dockerfile, fluent.conf and write_logs.sh in one directory
Dockerfile
FROM fluent/fluentd:v1.1.0
COPY ./fluent.conf /fluentd/etc/fluent.conf
COPY ./write_logs.sh /usr/local/bin/write-logs
RUN chmod 755 /usr/local/bin/write-logs
RUN find . -name in_tail.rb | xargs sed -i '/line.chomp!/a \ \ \ \ \ \ \ \ puts Time.now.to_datetime.iso8601(3)'
fluent.conf
write_logs.sh
!/bin/sh
counter=0
while true; do
sleep 0.01
echo $counter >> /tmp/test.log
counter=expr $counter + 1
done
2. run `docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem`
3. run `docker exec -it inotify_problem write-logs`
As you can see, logs are being polled with 5 sec frequency (`convert_line_to_event` function is executed every ~5 sec).
Now change `fluent/fluentd:v1.1.0` to `fluent/fluentd:v1.1.0-debian` in the Dockerfile and run:
1. `docker stop inotify_problem; docker rm inotify_problem; docker rmi inotify_problem`
2. `docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem`
3. `docker exec -it inotify_problem write-logs`
Observe that `convert_line_to_event` function is executed every ~0.01 sec.
This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 7 days
Description
When I use the
fluent/fluentd:v1.1.0
image thein_tail
plugin fallbacks to polling file every 5 seconds instead of usinginotify
. This causes loss in timestamp precision.fluent/fluentd:v1.1.0-debian
image handles watching file correctly and does not fallback to polling.Steps to reproduce
Dockerfile
,fluent.conf
andwrite_logs.sh
in one directory DockerfileCOPY ./fluent.conf /fluentd/etc/fluent.conf COPY ./write_logs.sh /usr/local/bin/write-logs RUN chmod 755 /usr/local/bin/write-logs
RUN find . -name in_tail.rb | xargs sed -i '/line.chomp!/a \ \ \ \ \ \ \ \ puts Time.now.to_datetime.iso8601(3)'
!/bin/sh
counter=0 while true; do sleep 0.01 echo $counter >> /tmp/test.log counter=
expr $counter + 1
done