search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.53k stars 430 forks source link

Acquis folder/* can trigger CrowdSec process fatal error => crash #1839

Closed rr404 closed 1 year ago

rr404 commented 1 year ago

What happened?

CrowdSec agent process crashes if it can't tail a file:

crowdsec.service - Crowdsec agent
   Loaded: loaded (/usr/lib/systemd/system/crowdsec.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2022-10-26 08:14:18 UTC; 17min ago
  Process: 13016 ExecStart=/usr/bin/crowdsec -c /etc/crowdsec/config.yaml (code=exited, status=1/FAILURE)
  Process: 13003 ExecStartPre=/usr/bin/crowdsec -c /etc/crowdsec/config.yaml -t (code=exited, status=0/SUCCESS)
 Main PID: 13016 (code=exited, status=1/FAILURE)

Oct 26 08:09:06 myhost.com systemd[1]: Starting Crowdsec agent...
Oct 26 08:09:07 myhost.com crowdsec[13003]: time="2022-10-26T08:09:07Z" level=warning msg="Deprecation warning: the pid_dir config can be safely removed and is not required"
Oct 26 08:09:09 myhost.com crowdsec[13016]: time="2022-10-26T08:09:09Z" level=warning msg="Deprecation warning: the pid_dir config can be safely removed and is not required"
Oct 26 08:09:10 myhost.com systemd[1]: Started Crowdsec agent.
Oct 26 08:14:18 myhost.com crowdsec[13016]: time="26-10-2022 08:14:18" level=fatal msg="starting acquisition error : tail for /var/log/apache2/domlogs/ftpxferlog.offse...k is empty"
Oct 26 08:14:18 myhost.com systemd[1]: crowdsec.service: main process exited, code=exited, status=1/FAILURE
Oct 26 08:14:18 myhost.com systemd[1]: Unit crowdsec.service entered failed state.
Oct 26 08:14:18 myhost.com systemd[1]: crowdsec.service failed.

This happened if the file has been deleted after the agent detected it and before the tail happens

When acquis is on a folder/ it happens Use case is for WHM filenames: Apache acquis for files is `- /var/log/apache2/domlogs/` and some random files pop-inAndOut of this folder triggering the acquisition crash

What did you expect to happen?

Agent should handle the error and not crash

How can we reproduce it (as minimally and precisely as possible)?

have an acquis file like "folder/*" run crowdsec make a script creating and then immediately deleting them

Anything else we need to know?

No response

Crowdsec version

2022/10/26 08:39:38 version: v1.4.1-el7-rpm-e1954adc325baa9e3420c324caabd50b7074dd77
2022/10/26 08:39:38 Codename: alphaga
2022/10/26 08:39:38 BuildDate: 2022-07-25_09:53:20
2022/10/26 08:39:38 GoVersion: 1.17.5
2022/10/26 08:39:38 Platform: linux
2022/10/26 08:39:38 Constraint_parser: >= 1.0, <= 2.0
2022/10/26 08:39:38 Constraint_scenario: >= 1.0, < 3.0
2022/10/26 08:39:38 Constraint_api: v1
2022/10/26 08:39:38 Constraint_acquis: >= 1.0, < 2.0

OS version

NAME="CloudLinux"
VERSION="7.8 (Alexei Leonov)"
ID="cloudlinux"
ID_LIKE="rhel fedora centos"
VERSION_ID="7.8"
PRETTY_NAME="CloudLinux 7.8 (Alexei Leonov)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:cloudlinux:cloudlinux:7.8:GA:server"
HOME_URL="https://www.cloudlinux.com/"
BUG_REPORT_URL="https://www.cloudlinux.com/support"

Enabled collections and parsers

```console crowdsecurity/apache2,enabled,0.1,apache2 support : parser and generic http scenarios ,collections crowdsecurity/base-http-scenarios,enabled,0.6,http common : scanners detection,collections crowdsecurity/cpanel,enabled,0.2,cpanel support : parser and bruteforce detection,collections crowdsecurity/http-cve,enabled,1.1,,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/mysql,enabled,0.1,mysql support : logs and brute-force scenarios,collections crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections fulljackz/pureftpd,enabled,0.1,Pureftpd support : parser for brute force detection on Pureftpd,collections crowdsecurity/apache2-logs,enabled,1.0,Parse Apache2 access and error logs,parsers crowdsecurity/cpanel-logs,enabled,0.4,Parse Cpanel logs,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,0.8,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/mysql-logs,enabled,0.3,Parse MySQL logs,parsers crowdsecurity/sshd-logs,enabled,1.9,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers fulljackz/pureftpd-logs,enabled,0.1,Parse pureftpd logs for bruteforce attempts,parsers crowdsecurity/CVE-2022-37042,enabled,0.1,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/cpanel-bf,enabled,0.2,Detect bruteforce on cpanel login,scenarios crowdsecurity/cpanel-bf-attempt,enabled,0.1,Detect bruteforce attempt on cpanel login,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.2,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.2,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.2,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/mysql-bf,enabled,0.1,Detect mysql bruteforce,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios fulljackz/pureftpd-bf,enabled,0.1,Detect pureftpd bruteforce,scenarios ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios ```

Acquisition config

```console acquis.d/apache.yaml filenames: - /var/log/apache2/domlogs/* labels: type: apache2 [root@cpanel-001-dev-fra crowdsec]# cat acquis.d/apache.yaml filenames: - /var/log/apache2/domlogs/* labels: type: apache2 [root@cpanel-001-dev-fra crowdsec]# cat acquis.yaml #Generated acquisition file - wizard.sh (service: apache2) / files : #journalctl_filter: # - _SYSTEMD_UNIT=apache2.service #labels: # type: apache2 --- #Generated acquisition file - wizard.sh (service: sshd) / files : /var/log/secure filenames: - /var/log/secure labels: type: syslog --- #Generated acquisition file - wizard.sh (service: mysql) / files : journalctl_filter: - _SYSTEMD_UNIT=mysql.service labels: type: mysql --- #Generated acquisition file - wizard.sh (service: linux) / files : /var/log/messages filenames: - /var/log/messages labels: type: syslog --- ``` ```console acquis.d/apache.yaml filenames: - /var/log/apache2/domlogs/* labels: type: apache2 ```

Config show

```console $ cscli config show Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log/ - Log level : info - Log Media : file Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 - Acquisition Folder : /etc/crowdsec/acquis.d cscli: - Output : human - Hub Branch : - Hub Folder : /etc/crowdsec/hub Local API Server: - Listen URL : 127.0.0.1:8080 - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - 127.0.0.1 - ::1 - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000 ```
mmetc commented 1 year ago

Bug is fixed, tests are missing, but I have a pending refactoring (extract a static validation method from the Configure method) before doing this one.