search

fangli / fluent-plugin-influxdb

A buffered output plugin for fluentd and InfluxDB
MIT License
111 stars 65 forks source link

Connecting to database : no timeout ? #94

Open RemiSlv opened 5 years ago

RemiSlv commented 5 years ago

Hi, I am facing an issue when fluentd is starting and tries to connect to an unavailable InfluxDB. Last trace from log is "Connecting to database".

If the DB is unavailable I would like fluentd to open its http source and store items in disk buffer.

Is that possible ?

2018-12-22 01:45:55 +0800 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2018-12-22 01:45:55 +0800 [info]: using configuration file: <ROOT>
  <system>
    log_level debug
  </system>
  <source>
    @type http
    @id input_http
    port 8888
  </source>
  <filter mkd.**>
    @type record_transformer
    remove_keys _time
    <record>
      time ${record["_time"]}
    </record>
  </filter>
  <match mkd.dev.depth>
    @type influxdb
    host "analytics.local"
    port 8087
    dbname "dev_mkd"
    measurement "depth"
    tag_keys ["e","s","d","t"]
    time_precision "ns"
    <buffer>
      @type "file"
      path "/var/log/td-agent/buffer/mkd.dev.depth"
      flush_interval 10s
      queued_chunks_limit_size 100
    </buffer>
  </match>
  <match mkd.**>
    @type stdout
    @id output_stdout
  </match>
</ROOT>
2018-12-22 01:45:55 +0800 [info]: starting fluentd-1.2.6 pid=22069 ruby="2.4.4"
2018-12-22 01:45:55 +0800 [info]: spawn command to main:  cmdline=["/opt/td-agent/embedded/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/td-agent/embedded/bin/fluentd", "--log", "/var/log/td-agent/td-agent.log", "--daemon", "/var/run/td-agent/td-agent.pid", "--under-supervisor"]
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-elasticsearch' version '2.11.11'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-influxdb' version '1.0.0'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-kafka' version '0.7.9'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-record-modifier' version '1.1.0'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.1.0'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-s3' version '1.1.6'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-td' version '1.0.0'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.4'
2018-12-22 01:45:55 +0800 [info]: gem 'fluent-plugin-webhdfs' version '1.2.3'
2018-12-22 01:45:55 +0800 [info]: gem 'fluentd' version '1.2.6'
2018-12-22 01:45:55 +0800 [info]: adding filter pattern="mkd.**" type="record_transformer"
2018-12-22 01:45:55 +0800 [info]: adding match pattern="mkd.dev.depth" type="influxdb"
2018-12-22 01:45:55 +0800 [info]: adding match pattern="mkd.**" type="stdout"
2018-12-22 01:45:55 +0800 [info]: adding source type="http"
2018-12-22 01:45:55 +0800 [info]: #0 starting fluentd worker pid=22086 ppid=22081 worker=0
2018-12-22 01:45:55 +0800 [debug]: #0 buffer started instance=70077661079240 stage_size=1275 queue_size=0
2018-12-22 01:45:55 +0800 [info]: #0 Connecting to database: dev_mkd, host: analytics.local, port: 8087, username: root, use_ssl = false, verify_ssl = true