search

timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
17.4k stars 872 forks source link

[Enhancement]: Tune down logging " refreshing continuous aggregate" #3892

Open miohtama opened 2 years ago

miohtama commented 2 years ago

What type of enhancement is this?

User experience

What subsystems and features will be improved?

Other

What does the enhancement do?

I am extensively using TimescaleDB continuous aggregate jobs.

Currently, my PostgreSQL log is populated with lines like (several lines per minute):

2021-12-02 17:24:56.273 UTC [18963] LOG:  refreshing continuous aggregate "candle_ohlcvx_aggregate_5m" in window [ 2021-11-29 17:25:00, 2021-12-02 17:20:00 ]

This is because there is a log every time a job runs and I have few jobs running every minute. This logging is not useful - continuous aggregate view refreshing log is only important if it somehow fails.

According to StackOverflow discussion this log level may not be configurable.

I would like to make it possible to tune down this log line, so that logs are not polluted.

Implementation challenges

I am not an expert on PSQL logging subsystem, so I have no idea how this change could be implemented.

bendoerr commented 2 years ago

HA, We have continuous aggregate refreshes hanging and I wish they were more verbose!

miohtama commented 2 years ago

I think they only warrant a notification or action after a certain timeout. Similar to slow query logging. Usually people are not logging every query either 😄

MKrkkl commented 1 year ago

Would it be possible to have it changed to NOTICE and in case of any issue WARN?

ATM refreshing continuous aggregate prints cover 25-75% of my logs coming from TSDB. If CAGGs are working as expected I don't wanna see any logs =)