search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.64k stars 446 forks source link

segmentation violation on notification command #1696

Closed gerbier closed 2 years ago

gerbier commented 2 years ago

What happened?

I'm running crowdsec version v1.4.1-e1954adc325baa9e3420c324caabd50b7074dd77 on ubuntu 22.04 and the command "`cscli notifications list" fails with the following error

root@pxcti1:~# cscli notifications list panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x1083502]

goroutine 1 [running]: main.getNotificationsConfiguration.func1({0x0, 0x0}, {0x0, 0x0}, {0xc0006a73e0, 0x41c645}) /home/runner/work/crowdsec/crowdsec/cmd/crowdsec-cli/notifications.go:144 +0x42 path/filepath.Walk({0x0, 0x0}, 0xc0006a7498) /opt/hostedtoolcache/go/1.17.12/x64/src/path/filepath/path.go:515 +0x50 main.getNotificationsConfiguration() /home/runner/work/crowdsec/crowdsec/cmd/crowdsec-cli/notifications.go:156 +0xda main.NewNotificationsCmd.func2(0xc00066a780, {0x1f430b8, 0x0, 0x0}) /home/runner/work/crowdsec/crowdsec/cmd/crowdsec-cli/notifications.go:51 +0x2e github.com/spf13/cobra.(Command).execute(0xc00066a780, {0x1f430b8, 0x0, 0x0}) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:860 +0x5f8 github.com/spf13/cobra.(Command).ExecuteC(0xc0002be280) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3bc github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902 main.main() /home/runner/work/crowdsec/crowdsec/cmd/crowdsec-cli/main.go:204 +0xad0

What did you expect to happen?

not an error

How can we reproduce it (as minimally and precisely as possible)?

cscli notifications list

Anything else we need to know?

No response

Crowdsec version

```console $ cscli version # paste output here root@pxcti1:~# cscli version 2022/08/01 13:43:33 version: v1.4.1-e1954adc325baa9e3420c324caabd50b7074dd77 2022/08/01 13:43:33 Codename: alphaga 2022/08/01 13:43:33 BuildDate: 2022-07-25_08:15:23 2022/08/01 13:43:33 GoVersion: 1.17.12 2022/08/01 13:43:33 Platform: linux 2022/08/01 13:43:33 Constraint_parser: >= 1.0, <= 2.0 2022/08/01 13:43:33 Constraint_scenario: >= 1.0, < 3.0 2022/08/01 13:43:33 Constraint_api: v1 2022/08/01 13:43:33 Constraint_acquis: >= 1.0, < 2.0

OS version

```console # On Linux: $ cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.1 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.1 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy $ uname -a Linux pxcti1 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux # On Windows: C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture # paste output here ```

Enabled collections and parsers

```console $ cscli hub list -o raw # paste output here crowdsecurity/apache2,enabled,0.1,apache2 support : parser and generic http scenarios ,collections crowdsecurity/base-http-scenarios,enabled,0.6,http common : scanners detection,collections crowdsecurity/http-cve,enabled,1.0,,collections crowdsecurity/iptables,enabled,0.1,iptables support : logs and port-scans detection scenarios,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/mysql,enabled,0.1,mysql support : logs and brute-force scenarios,collections crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections crowdsecurity/vsftpd,enabled,0.1,VSFTPD support : logs and brute-force scenarios,collections crowdsecurity/whitelist-good-actors,enabled,0.1,Good actors whitelists,collections crowdsecurity/apache2-logs,enabled,1.0,Parse Apache2 access and error logs,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,0.8,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/iptables-logs,enabled,0.3,Parse iptables drop logs,parsers crowdsecurity/mysql-logs,enabled,0.3,Parse MySQL logs,parsers crowdsecurity/sshd-logs,enabled,1.9,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/vsftpd-logs,enabled,0.2,Parse VSFTPD logs,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.2,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.2,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.2,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/iptables-scan-multi_ports,enabled,0.1,ban IPs that are scanning us,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/mysql-bf,enabled,0.1,Detect mysql bruteforce,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios crowdsecurity/vsftpd-bf,enabled,0.1,Detect FTP bruteforce (vsftpd),scenarios ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios crowdsecurity/cdn-whitelist,enabled,0.4,Whitelist CDN providers,postoverflows crowdsecurity/rdns,enabled,0.2,Lookup the DNS associated to the source IP only for overflows,postoverflows crowdsecurity/seo-bots-whitelist,enabled,0.4,Whitelist good search engine crawlers,postoverflows

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* # paste output here #Generated acquisition file - wizard.sh (service: linux) / files : /var/log/syslog /var/log/kern.log filenames: - /var/log/syslog - /var/log/kern.log labels: type: syslog --- cat: '/etc/crowdsec/acquis.d/*': Aucun fichier ou dossier de ce type # On Windows: C:\> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml # paste output here

Config show

```console $ cscli config show # paste output here Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log/ - Log level : info - Log Media : file Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 cscli: - Output : human - Hub Branch : master - Hub Folder : /etc/crowdsec/hub Local API Server: - Listen URL : localhost:8080 - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000

Prometheus metrics

```console $ cscli metrics # paste output here INFO[01-08-2022 01:47:55 PM] Acquisition Metrics: +----------------------+------------+--------------+----------------+------------------------+ | SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET | +----------------------+------------+--------------+----------------+------------------------+ | file:/var/log/syslog | 25 | - | 25 | - | +----------------------+------------+--------------+----------------+------------------------+ INFO[01-08-2022 01:47:55 PM] Parser Metrics: +---------------------------------+------+--------+----------+ | PARSERS | HITS | PARSED | UNPARSED | +---------------------------------+------+--------+----------+ | child-crowdsecurity/syslog-logs | 25 | 25 | - | | crowdsecurity/syslog-logs | 25 | 25 | - | +---------------------------------+------+--------+----------+ INFO[01-08-2022 01:47:55 PM] Local Api Metrics: +----------------------+--------+------+ | ROUTE | METHOD | HITS | +----------------------+--------+------+ | /v1/alerts | GET | 3 | | /v1/decisions/stream | GET | 105 | | /v1/heartbeat | GET | 19 | | /v1/watchers/login | POST | 5 | +----------------------+--------+------+ INFO[01-08-2022 01:47:55 PM] Local Api Machines Metrics: +----------------------------------+---------------+--------+------+ | MACHINE | ROUTE | METHOD | HITS | +----------------------------------+---------------+--------+------+ | 88cecf923c7f4189963bf9fb2a06da08 | /v1/alerts | GET | 3 | | 88cecf923c7f4189963bf9fb2a06da08 | /v1/heartbeat | GET | 19 | +----------------------------------+---------------+--------+------+ INFO[01-08-2022 01:47:55 PM] Local Api Bouncers Metrics: +------------------------------+----------------------+--------+------+ | BOUNCER | ROUTE | METHOD | HITS | +------------------------------+----------------------+--------+------+ | cs-firewall-bouncer-lo1CPnKO | /v1/decisions/stream | GET | 105 | +------------------------------+----------------------+--------+------+ INFO[01-08-2022 01:47:55 PM] Local Api Decisions: +--------------------------------------------+--------+--------+-------+ | REASON | ORIGIN | ACTION | COUNT | +--------------------------------------------+--------+--------+-------+ | crowdsecurity/http-crawl-non_statics | CAPI | ban | 23 | | crowdsecurity/http-cve-2021-41773 | CAPI | ban | 6 | | crowdsecurity/http-sensitive-files | CAPI | ban | 331 | | crowdsecurity/iptables-scan-multi_ports | CAPI | ban | 378 | | crowdsecurity/mysql-bf | CAPI | ban | 63 | | crowdsecurity/ssh-slow-bf | CAPI | ban | 7838 | | crowdsecurity/grafana-cve-2021-43798 | CAPI | ban | 1 | | crowdsecurity/http-backdoors-attempts | CAPI | ban | 367 | | crowdsecurity/thinkphp-cve-2018-20062 | CAPI | ban | 325 | | crowdsecurity/http-bad-user-agent | CAPI | ban | 1467 | | crowdsecurity/ssh-bf | CAPI | ban | 1603 | | crowdsecurity/f5-big-ip-cve-2020-5902 | CAPI | ban | 9 | | crowdsecurity/fortinet-cve-2018-13379 | CAPI | ban | 25 | | crowdsecurity/http-generic-bf | CAPI | ban | 7 | | crowdsecurity/http-path-traversal-probing | CAPI | ban | 803 | | crowdsecurity/http-probing | CAPI | ban | 1398 | | crowdsecurity/spring4shell_cve-2022-22965 | CAPI | ban | 1 | | crowdsecurity/vsftpd-bf | CAPI | ban | 2 | | crowdsecurity/apache_log4j2_cve-2021-44228 | CAPI | ban | 35 | | crowdsecurity/http-cve-2021-42013 | CAPI | ban | 2 | | crowdsecurity/http-open-proxy | CAPI | ban | 475 | | ltsich/http-w00tw00t | CAPI | ban | 3 | +--------------------------------------------+--------+--------+-------+

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

LaurenceJJones commented 2 years ago

Hey thank you for bringing this to our attention. Do you have any notifications currently setup? Could you provide me your profiles.yaml? /etc/crowdsec/profiles.yaml

Edit: Adding debugging information: Current default profiles.yaml provides a notification but it is commented out so yaml is unmarshalled it will not exists on the struct https://github.com/crowdsecurity/crowdsec/blob/c7422420944e58498665e895a496bdaeb3fdbcb4/config/profiles.yaml#L9

The function getNotificationsConfiguration loops over profile.Notifications but this may be nil https://github.com/crowdsecurity/crowdsec/blob/c7422420944e58498665e895a496bdaeb3fdbcb4/cmd/crowdsec-cli/notifications.go#L164

gerbier commented 2 years ago

when I had the error, I had no notifications configured at all

but now I have added email_default notifications, and I do not have the problem any more root@pxcti1:~# cscli notifications list

NAME TYPE PROFILE NAME

email_default email default_ip_remediation

ps : the doc site https://docs.crowdsec.net/docs/notification_plugins/intro/ does not mention the email_default plugin