fangli / fluent-plugin-influxdb

A buffered output plugin for fluentd and InfluxDB
MIT License
111 stars 65 forks source link

Buffers are not flushed after td-agent restart #63

Closed ton31337 closed 7 years ago

ton31337 commented 7 years ago

After td-agent restart these are not flushed to InfluxDB and InfluxDB returns response with code 400 (Bad Request). Removing these buffer's files and restarting td-agent, everything is fine, InfluxDB returns 204.

[~]# ls -la /var/log/td-agent/buffer/
total 163852
drwxr-xr-x 2 td-agent td-agent    4096 Nov 17 14:29 .
drwxr-xr-x 3 td-agent td-agent    4096 Nov 17 03:20 ..
-rw-r--r-- 1 td-agent td-agent 6204727 Nov 17 14:36 openresty.buffer..b541800455821e280.log
-rw-r--r-- 1 td-agent td-agent   68346 Nov 17 11:47 openresty.buffer..q5417dc17eb4c3ac2.log
-rw-r--r-- 1 td-agent td-agent 8388194 Nov 17 11:55 openresty.buffer..q5417dc1cb04ba5c3.log
-rw-r--r-- 1 td-agent td-agent 8388383 Nov 17 12:02 openresty.buffer..q5417ddee752d0acd.log
-rw-r--r-- 1 td-agent td-agent 8388508 Nov 17 12:11 openresty.buffer..q5417df82b373f84c.log
-rw-r--r-- 1 td-agent td-agent 8388184 Nov 17 12:21 openresty.buffer..q5417e17983be7504.log
-rw-r--r-- 1 td-agent td-agent 8387911 Nov 17 12:30 openresty.buffer..q5417e3b708d850da.log
-rw-r--r-- 1 td-agent td-agent 8388467 Nov 17 12:40 openresty.buffer..q5417e5c5915f2753.log
-rw-r--r-- 1 td-agent td-agent 8387715 Nov 17 12:49 openresty.buffer..q5417e7ea0f3f354b.log
-rw-r--r-- 1 td-agent td-agent 8387881 Nov 17 12:57 openresty.buffer..q5417e9e221e86eb0.log
-rw-r--r-- 1 td-agent td-agent 8388236 Nov 17 13:06 openresty.buffer..q5417ebc5ac077d42.log
-rw-r--r-- 1 td-agent td-agent 8388493 Nov 17 13:15 openresty.buffer..q5417edd2c3c807b6.log
-rw-r--r-- 1 td-agent td-agent 8388199 Nov 17 13:24 openresty.buffer..q5417efca0eb5e84f.log
-rw-r--r-- 1 td-agent td-agent 8388511 Nov 17 13:34 openresty.buffer..q5417f1b799fed6ab.log
-rw-r--r-- 1 td-agent td-agent 8388219 Nov 17 13:43 openresty.buffer..q5417f3e4ecafe8e4.log
-rw-r--r-- 1 td-agent td-agent 8388417 Nov 17 13:49 openresty.buffer..q5417f5f0e5c500cf.log
-rw-r--r-- 1 td-agent td-agent 8388300 Nov 17 13:57 openresty.buffer..q5417f759ad405209.log
-rw-r--r-- 1 td-agent td-agent 8388106 Nov 17 14:05 openresty.buffer..q5417f93bb3a68ac4.log
-rw-r--r-- 1 td-agent td-agent 8388072 Nov 17 14:12 openresty.buffer..q5417faf8a4073910.log
-rw-r--r-- 1 td-agent td-agent 8388408 Nov 17 14:19 openresty.buffer..q5417fc9077170e98.log
-rw-r--r-- 1 td-agent td-agent 8388289 Nov 17 14:29 openresty.buffer..q5417fe1919044b32.log

is this known bug or something is bad with configuration?

<match metrics.openresty>
  type influxdb
  host X.X.X.X
  port 8086
  user metrics
  password xxx
  dbname metrics
  time_key msec
  time_precision ms
  buffer_type file
  buffer_path /var/log/td-agent/buffer/openresty.buffer
  buffer_queue_limit 512
  flush_interval 5s
</match>
repeatedly commented 7 years ago

If you didn't update influxdb, plugin and other middleware, you may send invalid value to influxdb. But I'm not sure when influxdb return 400...

ton31337 commented 7 years ago

Looks like problem was in InfluxDB