Logrotation

[katzeprior]

What happened?

Every midnight my server puts crowdsec.log into crowdsec.log.1.gz and creates a new crowdsec.log file but doesn't write any logs to it, only after a restart of crowdsec it starts writing again.

What did you expect to happen?

I expected it to don't do logrotation because I never noticed it before or for it to write to the new log file.

How can we reproduce it (as minimally and precisely as possible)?

I have no idea, hard to test because it only happens at midnight.

Anything else we need to know?

No response

Crowdsec version

```console $ cscli version 2022/09/03 09:06:34 version: v1.4.1-debian-pragmatic-e1954adc325baa9e3420c324caabd50b7074dd77 2022/09/03 09:06:34 Codename: alphaga 2022/09/03 09:06:34 BuildDate: 2022-07-25_09:20:06 2022/09/03 09:06:34 GoVersion: 1.17.5 2022/09/03 09:06:34 Platform: linux 2022/09/03 09:06:34 Constraint_parser: >= 1.0, <= 2.0 2022/09/03 09:06:34 Constraint_scenario: >= 1.0, < 3.0 2022/09/03 09:06:34 Constraint_api: v1 2022/09/03 09:06:34 Constraint_acquis: >= 1.0, < 2.0 ```

OS version

```console $ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" $ uname -a Linux joshaprior 5.10.0-17-amd64 #1 SMP Debian 5.10.136-1 (2022-08-13) x86_64 GNU/Linux ```

Enabled collections and parsers

```console $ cscli hub list -o raw crowdsecurity/base-http-scenarios,enabled,0.6,http common : scanners detection,collections crowdsecurity/endlessh,enabled,0.1,endlessh support : logs parser and brute-force detection,collections crowdsecurity/http-cve,enabled,1.0,,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/linux-lpe,enabled,0.1,Linux Local Privilege Escalation collection : detect trivial LPEs,collections crowdsecurity/nginx,enabled,0.2,nginx support : parser and generic http scenarios,collections crowdsecurity/nginx-proxy-manager,enabled,0.1,Nginx Proxy Manager support : parser and generic http scenarios,collections crowdsecurity/pgsql,enabled,0.1,postgres support : logs and brute-force scenarios,collections crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/endlessh-logs,enabled,0.1,Parse Endlessh logs,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,0.8,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/nginx-logs,enabled,1.2,Parse nginx access and error logs,parsers crowdsecurity/nginx-proxy-manager-logs,enabled,0.2,Parse Nginx Proxy Manager access and error logs,parsers crowdsecurity/pgsql-logs,enabled,0.5,Parse PgSQL logs,parsers crowdsecurity/pkexec-logs,enabled,0.1,Parse pkexec logs specifically for CVE-2021-4034,parsers crowdsecurity/sshd-logs,enabled,1.9,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers crowdsecurity/CVE-2021-4034,enabled,0.1,Detect CVE-2021-4034 exploits,scenarios crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/endlessh-bf,enabled,0.1,Detect SSH bruteforce caught by Endlessh,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.2,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.2,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.2,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/nginx-req-limit-exceeded,enabled,0.1,Detects IPs which violate nginx's user set request limit.,scenarios crowdsecurity/pgsql-bf,enabled,0.1,Detect PgSQL bruteforce,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios timokoessler/uptime-kuma-bf,enabled,0.1,Detect Uptime Kuma bruteforce,scenarios ```

Acquisition config

```console $ cat /etc/crowdsec/acquis.yaml #Generated acquisition file - wizard.sh (service: nginx) / files : /var/log/nginx/access.log /var/log/nginx/error.log filenames: - /var/log/nginx/access.log - /var/log/nginx/error.log labels: type: nginx --- #Generated acquisition file - wizard.sh (service: sshd) / files : /var/log/auth.log filenames: - /var/log/auth.log labels: type: syslog --- #Generated acquisition file - wizard.sh (service: mysql) / files : journalctl_filter: - _SYSTEMD_UNIT=mysql.service labels: type: mysql --- #Generated acquisition file - wizard.sh (service: linux) / files : /var/log/syslog /var/log/kern.log /var/log/messages filenames: - /var/log/syslog - /var/log/kern.log - /var/log/messages labels: type: syslog --- source: docker container_name: - uptime-kuma labels: type: uptime-kuma --- source: docker container_name: - endlessh labels: type: endlessh --- ```

Config show

```console $ cscli config show Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log/ - Log level : info - Log Media : file Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 cscli: - Output : human - Hub Branch : - Hub Folder : /etc/crowdsec/hub Local API Server: - Listen URL : 127.0.0.1:8080 - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - 127.0.0.1 - ::1 - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000 ```

Prometheus metrics

```console $ cscli metrics INFO[03-09-2022 09:11:14 AM] Buckets Metrics: +--------------------------------------+---------------+-----------+--------------+--------+---------+ | BUCKET | CURRENT COUNT | OVERFLOWS | INSTANTIATED | POURED | EXPIRED | +--------------------------------------+---------------+-----------+--------------+--------+---------+ | crowdsecurity/endlessh-bf | 5 | - | 9 | 16 | 4 | | crowdsecurity/http-crawl-non_statics | - | - | 21 | 29 | 21 | +--------------------------------------+---------------+-----------+--------------+--------+---------+ INFO[03-09-2022 09:11:14 AM] Acquisition Metrics: +---------------------------------------------------+------------+--------------+----------------+------------------------+ | SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET | +---------------------------------------------------+------------+--------------+----------------+------------------------+ | docker:endlessh | 32 | 16 | 16 | 16 | | docker:uptime-kuma | 1 | - | 1 | - | | file:/var/log/auth.log | 30 | - | 30 | - | | file:/var/log/kern.log | 451 | - | 451 | - | | file:/var/log/messages | 451 | - | 451 | - | | file:/var/log/nginx/access.log | 343 | 335 | 8 | 29 | | file:/var/log/nginx/error.log | 4 | 2 | 2 | - | | file:/var/log/syslog | 3.05k | - | 3.05k | - | | journalctl:journalctl-_SYSTEMD_UNIT=mysql.service | 1 | - | 1 | - | +---------------------------------------------------+------------+--------------+----------------+------------------------+ INFO[03-09-2022 09:11:14 AM] Parser Metrics: +-----------------------------------+-------+--------+----------+ | PARSERS | HITS | PARSED | UNPARSED | +-----------------------------------+-------+--------+----------+ | child-crowdsecurity/endlessh-logs | 48 | 16 | 32 | | child-crowdsecurity/http-logs | 1.01k | 522 | 489 | | child-crowdsecurity/nginx-logs | 359 | 337 | 22 | | child-crowdsecurity/sshd-logs | 50 | - | 50 | | child-crowdsecurity/syslog-logs | 3.98k | 3.98k | - | | crowdsecurity/dateparse-enrich | 353 | 353 | - | | crowdsecurity/endlessh-logs | 32 | 16 | 16 | | crowdsecurity/geoip-enrich | 353 | 353 | - | | crowdsecurity/http-logs | 337 | 163 | 174 | | crowdsecurity/nginx-logs | 347 | 337 | 10 | | crowdsecurity/non-syslog | 381 | 381 | - | | crowdsecurity/sshd-logs | 5 | - | 5 | | crowdsecurity/syslog-logs | 3.98k | 3.98k | - | | crowdsecurity/whitelists | 353 | 353 | - | +-----------------------------------+-------+--------+----------+ INFO[03-09-2022 09:11:14 AM] Local Api Metrics: +----------------------+--------+------+ | ROUTE | METHOD | HITS | +----------------------+--------+------+ | /v1/decisions/stream | GET | 331 | | /v1/heartbeat | GET | 55 | | /v1/watchers/login | POST | 2 | +----------------------+--------+------+ INFO[03-09-2022 09:11:14 AM] Local Api Machines Metrics: +--------------------------------------------------+---------------+--------+------+ | MACHINE | ROUTE | METHOD | HITS | +--------------------------------------------------+---------------+--------+------+ | cdc028b6ed9949249b893deb59b3b0acMt6eX9sLpNW0HLmG | /v1/heartbeat | GET | 55 | +--------------------------------------------------+---------------+--------+------+ INFO[03-09-2022 09:11:14 AM] Local Api Bouncers Metrics: +----------------------------+----------------------+--------+------+ | BOUNCER | ROUTE | METHOD | HITS | +----------------------------+----------------------+--------+------+ | FirewallBouncer-1645702404 | /v1/decisions/stream | GET | 331 | +----------------------------+----------------------+--------+------+ INFO[03-09-2022 09:11:14 AM] Local Api Decisions: +--------------------------------------------+----------+--------+-------+ | REASON | ORIGIN | ACTION | COUNT | +--------------------------------------------+----------+--------+-------+ | crowdsecurity/thinkphp-cve-2018-20062 | CAPI | ban | 43 | | crowdsecurity/ssh-bf | CAPI | ban | 308 | | crowdsecurity/ssh-slow-bf | CAPI | ban | 9664 | | crowdsecurity/http-crawl-non_statics | CAPI | ban | 8 | | crowdsecurity/pgsql-bf | CAPI | ban | 94 | | crowdsecurity/http-backdoors-attempts | CAPI | ban | 5 | | crowdsecurity/http-bad-user-agent | CAPI | ban | 2193 | | crowdsecurity/http-bad-user-agent | crowdsec | ban | 1 | | crowdsecurity/http-path-traversal-probing | CAPI | ban | 67 | | crowdsecurity/http-sensitive-files | CAPI | ban | 7 | | crowdsecurity/nginx-req-limit-exceeded | CAPI | ban | 24 | | ltsich/http-w00tw00t | CAPI | ban | 7 | | crowdsecurity/apache_log4j2_cve-2021-44228 | CAPI | ban | 5 | | crowdsecurity/http-cve-2021-41773 | CAPI | ban | 1 | | crowdsecurity/http-generic-bf | CAPI | ban | 16 | | crowdsecurity/http-open-proxy | CAPI | ban | 74 | | crowdsecurity/http-open-proxy | crowdsec | ban | 6 | | crowdsecurity/http-probing | CAPI | ban | 3007 | | crowdsecurity/http-probing | crowdsec | ban | 1 | | crowdsecurity/endlessh-bf | CAPI | ban | 223 | | crowdsecurity/endlessh-bf | crowdsec | ban | 2 | | crowdsecurity/fortinet-cve-2018-13379 | CAPI | ban | 30 | +--------------------------------------------+----------+--------+-------+ INFO[03-09-2022 09:11:14 AM] Local Api Alerts: +---------------------------------------+-------+ | REASON | COUNT | +---------------------------------------+-------+ | crowdsecurity/endlessh-bf | 19 | | crowdsecurity/fortinet-cve-2018-13379 | 1 | | crowdsecurity/http-bad-user-agent | 6 | | crowdsecurity/http-open-proxy | 43 | | crowdsecurity/http-probing | 1 | | crowdsecurity/ssh-bf | 5 | | crowdsecurity/ssh-slow-bf | 19 | +---------------------------------------+-------+ ```

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

custom abuseipdb notification plugin i am working on. ```console type: abuseipdb # Don't change name: abuseipdb_default # Must match the registered plugin in the profile # One of "trace", "debug", "info", "warn", "error", "off" log_level: info # group_wait: # Time to wait collecting alerts before relaying a message to this plugin, eg "30s" # group_threshold: # Amount of alerts that triggers a message before has expired, eg "10" # max_retry: # Number of attempts to relay messages to plugins in case of error # timeout: # Time to wait for response from the plugin before considering the attempt a failure, eg "10s" #------------------------- # plugin-specific options # The following template receives a list of models.Alert objects # The output goes in the logs and to a text file, if defined format: | {{.|toJson}} # Api key for abuseipdb (https://www.abuseipdb.com/account/api) api_key: e459a0b931c780b6065f63555fd69a8741a0e43ed3caa8ab1113ed8c585418d573d32ee0799a7a40 # # output_file: # notifications will be appended here. optional --- # type: abuseipdb # name: abuseipdb_second_notification # ... ```

[buixor]

Hello @katzeprior !

Are you using crowdsec's built-in log rotation or the logrotate daemon from the system?

[katzeprior]

I have no idea, is there a way to check this?

[buixor]

you can check /etc/logrotate.d/ files if it's the logrotate from the system. If it's crowdsec doing its own logrotate, it will be in https://doc.crowdsec.net/docs/next/configuration/crowdsec_configuration#common by using the dedicated directives

[katzeprior]

@buixor Its the system doing the logrotate. Will close the issue because I will be reinstalling my system next vacation so its clean and will reopen if it ever happens again.

$ cat /etc/logrotate.d/crowdsec 
/var/log/crowdsec.log
/var/log/crowdsec_api.log
{
  rotate 4
  daily
  compress
  missingok
  notifempty
}