search

graphite-project / carbon

Carbon is one of the components of Graphite, and is responsible for receiving metrics over the network and writing them down to disk using a storage backend.
http://graphite.readthedocs.org/
Apache License 2.0
1.5k stars 490 forks source link

[Q] Set a custom log retention in carbon-relay #891

Closed tbenz9 closed 4 years ago

tbenz9 commented 4 years ago

Question: How do I configure log retention in Carbon relay?

Scenario: We only want to keep 30 days of carbon logs and delete anything older than 30 days. carbon.conf

/opt/graphite/conf# cat carbon.conf
[relay]
USER = graphite
ENABLE_LOGROTATION = True
...TRUNCATED...

I don't see any carbon or graphite files under /etc/logrotate.d/

Actual Behavior: We are getting several months worth of logs stored that are filling our disks. Logrotate does seem to be rotating the logs daily, but not cleaning up old log files.

/opt/graphite/storage/log/carbon-relay/carbon-relay-a# ll
total 2926240
drwxr-xr-x 2 graphite graphite    16384 Apr 14 15:30 ./
drwxr-xr-x 3 graphite graphite     4096 Sep 11  2018 ../
-rwxr-xr-x 1 graphite graphite     3200 Apr 13 06:43 clients.log*
-rwxr-xr-x 1 graphite graphite     5184 Dec 18 16:44 clients.log.2019_12_18*
-rwxr-xr-x 1 graphite graphite     5632 Feb 18 17:19 clients.log.2020_2_18*
-rwxr-xr-x 1 graphite graphite     6048 Feb 28 14:47 clients.log.2020_2_28*
-rwxr-xr-x 1 graphite graphite     8400 Mar  1 07:13 clients.log.2020_3_1*
-rwxr-xr-x 1 graphite graphite     4158 Mar 10 17:54 clients.log.2020_3_10*
-rwxr-xr-x 1 graphite graphite   194688 Mar 11 23:59 clients.log.2020_3_11*
-rwxr-xr-x 1 graphite graphite  3015250 Mar 12 23:59 clients.log.2020_3_12*
-rwxr-xr-x 1 graphite graphite  2942100 Mar 13 11:47 clients.log.2020_3_13*
-rwxr-xr-x 1 graphite graphite     5600 Mar  4 02:21 clients.log.2020_3_4*
-rwxr-xr-x 1 graphite graphite     5600 Mar  9 07:34 clients.log.2020_3_9*
-rwxr-xr-x 1 graphite graphite     9600 Apr 10 03:00 clients.log.2020_4_10*
-rwxr-xr-x 1 graphite graphite     1600 Apr 11 09:20 clients.log.2020_4_11*
-rwxr-xr-x 1 graphite graphite     3200 Apr  2 02:57 clients.log.2020_4_2*
-rwxr-xr-x 1 graphite graphite     6400 Apr  4 13:46 clients.log.2020_4_4*
-rwxr-xr-x 1 graphite graphite     3200 Apr  7 02:17 clients.log.2020_4_7*
-rwxr-xr-x 1 graphite graphite     9200 Apr  8 07:13 clients.log.2020_4_8*
-rwxr-xr-x 1 graphite graphite  2296265 Mar 13 11:47 console.log*
-rwxr-xr-x 1 graphite graphite     1953 Dec 18 16:44 console.log.2019_12_18*
-rwxr-xr-x 1 graphite graphite     2035 Feb 18 17:19 console.log.2020_2_18*
-rwxr-xr-x 1 graphite graphite     2117 Feb 28 14:47 console.log.2020_2_28*
-rwxr-xr-x 1 graphite graphite     1626 Mar 10 17:54 console.log.2020_3_10*
-rwxr-xr-x 1 graphite graphite   160652 Mar 11 23:59 console.log.2020_3_11*
-rwxr-xr-x 1 graphite graphite  2385884 Mar 12 23:59 console.log.2020_3_12*
-rwxr-xr-x 1 graphite graphite 20487564 Apr 14 15:43 listener.log*
-rwxr-xr-x 1 graphite graphite 27339671 Jan  1 23:59 listener.log.2020_1_1*
-rwxr-xr-x 1 graphite graphite 27380396 Jan 10 23:59 listener.log.2020_1_10*
-rwxr-xr-x 1 graphite graphite 27381281 Jan 11 23:59 listener.log.2020_1_11*
-rwxr-xr-x 1 graphite graphite 27638673 Jan 12 23:59 listener.log.2020_1_12*
-rwxr-xr-x 1 graphite graphite 27623640 Jan 13 23:59 listener.log.2020_1_13*
-rwxr-xr-x 1 graphite graphite 27645654 Jan 14 23:59 listener.log.2020_1_14*
-rwxr-xr-x 1 graphite graphite 27569494 Jan 15 23:59 listener.log.2020_1_15*
-rwxr-xr-x 1 graphite graphite 27744048 Jan 16 23:59 listener.log.2020_1_16*
-rwxr-xr-x 1 graphite graphite 27743370 Jan 17 23:59 listener.log.2020_1_17*
-rwxr-xr-x 1 graphite graphite 27745816 Jan 18 23:59 listener.log.2020_1_18*
-rwxr-xr-x 1 graphite graphite 27745652 Jan 19 23:59 listener.log.2020_1_19*
-rwxr-xr-x 1 graphite graphite 27374708 Jan  2 23:59 listener.log.2020_1_2*
-rwxr-xr-x 1 graphite graphite 27748224 Jan 20 23:59 listener.log.2020_1_20*
-rwxr-xr-x 1 graphite graphite 27799590 Jan 21 23:59 listener.log.2020_1_21*
-rwxr-xr-x 1 graphite graphite 27812721 Jan 22 23:59 listener.log.2020_1_22*
-rwxr-xr-x 1 graphite graphite 27799249 Jan 23 23:59 listener.log.2020_1_23*
-rwxr-xr-x 1 graphite graphite 27784625 Jan 24 23:59 listener.log.2020_1_24*
-rwxr-xr-x 1 graphite graphite 27770534 Jan 25 23:59 listener.log.2020_1_25*
-rwxr-xr-x 1 graphite graphite 27762983 Jan 26 23:59 listener.log.2020_1_26*
-rwxr-xr-x 1 graphite graphite 27785425 Jan 27 23:59 listener.log.2020_1_27*
-rwxr-xr-x 1 graphite graphite 27800284 Jan 28 23:59 listener.log.2020_1_28*
...TRUNCATED...
piotr1212 commented 4 years ago

I don't think you can do that with the internal logrotation. Disable it and enable logrotate from your os.