influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.59k stars 5.56k forks source link

make persistent connection to postgresql #1977

Closed wei-hai closed 6 years ago

wei-hai commented 7 years ago

Feature Request

Let Telegraf make persistent connection to postgresql database

Proposal:

We are using telegraf to collect metric data from postgresql and it works well, however, we found that it doesn’t keep persistent connection postgresql, which causes unnecessary growth and noises in postgresql log file

Current behavior:

Telegraf makes connection to postgresql every INTERVAL seconds and then disconnects.

Desired behavior:

Telegraf shall make persistent connection to postgresql

Use case: [Why is this important (helps with prioritizing requests)]

Too many connections and disconnections cause growth and noises in postgresql log files which is unnecessary at all.

Log output:

< 2016-10-31 10:48:14.863 PDT >LOG: disconnection: session time: 0:00:00.014 user=telegraf database=postgres host=127.0.0.1 port=36218 < 2016-10-31 10:48:15.849 PDT >LOG: connection received: host=127.0.0.1 port=36220 < 2016-10-31 10:48:15.850 PDT >LOG: connection authorized: user=telegraf database=postgres < 2016-10-31 10:48:15.863 PDT >LOG: disconnection: session time: 0:00:00.014 user=telegraf database=postgres host=127.0.0.1 port=36220

mnietz commented 7 years ago

:+1:

phemmer commented 7 years ago

I'll probably tackle this soon (within the next couple weeks). The way I think i'm going to do it is to add a pool_size_max = X parameter. If X is 0, the current behavior will be used (disconnect once done). Otherwise it'll connect up to X times, running the queries in parallel. To just have 1 long lived connection, it'd just be pool_size_max = 1.

james-lawrence commented 7 years ago

@phemmer may I suggest we just specify the 3 values for a standard db connection?

danielnelson commented 6 years ago

Closing, was fixed back in 1.6