From time to time, fluentd has problems to flush the buffer to elasticsearch. I have to delete all buffer files, restart fluentd and then it works again. I cannot see any problems on elasticsearch site ...
Also tried to move some buffer files out of the buffer directory (oldest ones) but doesn help - fluentd tries to process the newer ones but fails:
if i delete all buffer + meta files and restart the fluentd container, logs are forwarded correctly
if i copy one old buffer + meta file to the buffer directory, the same error like before is printed and logging is broken; new logs are written to new buffer files
tried to see any difference between new / old buffer files but didn't see any major differences (what would explain why some files cannot be forwarded but others can)
Problem
From time to time, fluentd has problems to flush the buffer to elasticsearch. I have to delete all buffer files, restart fluentd and then it works again. I cannot see any problems on elasticsearch site ... Also tried to move some buffer files out of the buffer directory (oldest ones) but doesn help - fluentd tries to process the newer ones but fails:
Steps to replicate
Expected Behavior or What you need to ask
Logs should be pushed correctly to elasticsearch.
Using Fluentd and ES plugin versions