search

awslabs / amazon-kinesis-agent

Continuously monitors a set of log files and sends new data to the Amazon Kinesis Stream and Amazon Kinesis Firehose in near-real-time.
Other
352 stars 218 forks source link

How does kinesis agent log getting logrotate? #249

Closed MrAtheist closed 1 year ago

MrAtheist commented 1 year ago

Hello,

How does the kinesis agent log getting rotated in the first place? I've checked in various logrotate conf but the only reference is buried under /etc/cron.d/aws-kinesis-agent

sh-4.2$ ls -la /var/log/aws-kinesis-agent

total 26404
drwxr-xr-x  2 root aws-kinesis-agent-user   12288 Dec  9 09:00 .
drwxr-xr-x 11 root root                      4096 Dec  9 03:22 ..
-rw-r--r--  1 root root                     27391 Dec  2 10:00 aws-kinesis-agent-2022-12-02-09.log.gz
-rw-r--r--  1 root root                     27050 Dec  2 11:00 aws-kinesis-agent-2022-12-02-10.log.gz
-rw-r--r--  1 root root                     26985 Dec  2 12:00 aws-kinesis-agent-2022-12-02-11.log.gz
-rw-r--r--  1 root root                     26693 Dec  2 13:00 aws-kinesis-agent-2022-12-02-12.log.gz
-rw-r--r--  1 root root                     26804 Dec  2 14:00 aws-kinesis-agent-2022-12-02-13.log.gz
-rw-r--r--  1 root root                     26930 Dec  2 15:00 aws-kinesis-agent-2022-12-02-14.log.gz
-rw-r--r--  1 root root                     26708 Dec  2 16:00 aws-kinesis-agent-2022-12-02-15.log.gz
-rw-r--r--  1 root root                     26786 Dec  2 17:00 aws-kinesis-agent-2022-12-02-16.log.gz
-rw-r--r--  1 root root                     26984 Dec  2 18:00 aws-kinesis-agent-2022-12-02-17.log.gz
-rw-r--r--  1 root root                   1697944 Dec  2 19:00 aws-kinesis-agent-2022-12-02-18.log.gz
....
# this seems to be rotated hourly, but nothing under `/etc/cron.hourly`...
# the only reference is under `/etc/cron.d`:

sh-4.2$ cat /etc/cron.d/aws-kinesis-agent
* * * * * root /usr/bin/aws-kinesis-agent-babysit > /dev/null 2>&1

# here's what /etc/cron* looks like...

sh-4.2$  ls -la /etc/cron.*
-rw------- 1 root root  0 Jan 16  2020 /etc/cron.deny

/etc/cron.d:
total 32
drwxr-xr-x  2 root root   98 Dec  9 09:56 .
drwxr-xr-x 93 root root 8192 Dec  9 09:58 ..
-rw-r--r--  1 root root  128 Jan 16  2020 0hourly
-rw-r--r--  1 root root   67 Dec 15  2021 aws-kinesis-agent
-rw-r--r--  1 root root  108 Aug  2  2018 raid-check
-rw-------  1 root root  235 Dec 17  2019 sysstat
-rw-r--r--  1 root root  194 Jun  8  2021 update-motd

/etc/cron.daily:
total 24
drwxr-xr-x  2 root root   57 Dec  9 09:50 .
drwxr-xr-x 93 root root 8192 Dec  9 09:58 ..
-rwx------  1 root root  219 Jul 27  2018 logrotate
-rwxr-xr-x  1 root root  618 Apr 29  2019 man-db.cron
-rwx------  1 root root  208 Jul 27  2018 mlocate

/etc/cron.hourly:
total 16
drwxr-xr-x  2 root root   22 Dec  9 10:00 .
drwxr-xr-x 93 root root 8192 Dec  9 09:58 ..
-rwxr-xr-x  1 root root  392 Jan 16  2020 0anacron

/etc/cron.monthly:
total 12
drwxr-xr-x  2 root root    6 Oct 18  2017 .
drwxr-xr-x 93 root root 8192 Dec  9 09:58 ..

/etc/cron.weekly:
total 12
drwxr-xr-x  2 root root    6 Oct 18  2017 .
drwxr-xr-x 93 root root 8192 Dec  9 09:58 ..

Is there anything else i'm missing here...? or does the kinesis agent itself runs the logrotate..?

MrAtheist commented 1 year ago

Closing this... as it turns out that aws-kinesis-agent has set up its own logrotate under /etc/aws-kinesis/log4j.xml...