search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.53k stars 430 forks source link

goroutine crowdsec/ShowPrometheus crashed #2755

Closed jstnhd1 closed 7 months ago

jstnhd1 commented 7 months ago

What happened?

running sudo cscli metrics shows crowdsec - goroutine crowdsec/ShowPrometheus crashed : protobuf tag not enough fields in MetricFamily.state: please report this error to https://github.com/crowdsecurity/crowdsec/

What did you expect to happen?

expected metrics

How can we reproduce it (as minimally and precisely as possible)?

fresh install of crowdsec with unifi collection, set aquis yaml as directed. no other changes to config.

Anything else we need to know?

thank you for your work, i appreciate y'all.

Crowdsec version

```console $ cscli version # 2024/01/18 02:37:31 version: v1.4.6-6+b3-debian 2024/01/18 02:37:31 Codename: alphaga 2024/01/18 02:37:31 BuildDate: 2023-10-18_17:20:28 2024/01/18 02:37:31 GoVersion: 1.21.3 2024/01/18 02:37:31 Platform: linux 2024/01/18 02:37:31 Constraint_parser: >= 1.0, <= 2.0 2024/01/18 02:37:31 Constraint_scenario: >= 1.0, < 3.0 2024/01/18 02:37:31 Constraint_api: v1 2024/01/18 02:37:31 Constraint_acquis: >= 1.0, < 2.0 ```

OS version

```console # On Linux: $ cat /etc/os-release # PRETTY_NAME="Kali GNU/Linux Rolling" NAME="Kali GNU/Linux" VERSION_ID="2023.4" VERSION="2023.4" VERSION_CODENAME=kali-rolling ID=kali ID_LIKE=debian HOME_URL="https://www.kali.org/" SUPPORT_URL="https://forums.kali.org/" BUG_REPORT_URL="https://bugs.kali.org/" ANSI_COLOR="1;31" $ uname -a # Linux kali 6.5.0-kali3-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.6-1kali1 (2023-10-09) x86_64 GNU/Linux ```

Enabled collections and parsers

```console $ cscli hub list -o raw # crowdsecurity/apache2,enabled,0.1,apache2 support : parser and generic http scenarios ,collections crowdsecurity/base-http-scenarios,enabled,0.6,http common : scanners detection,collections crowdsecurity/http-cve,enabled,1.9,,collections crowdsecurity/iptables,enabled,0.1,iptables support : logs and port-scans detection scenarios,collections crowdsecurity/linux,"enabled,tainted",?,core linux support : syslog+geoip+ssh,collections crowdsecurity/nginx,enabled,0.2,nginx support : parser and generic http scenarios,collections crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections crowdsecurity/unifi,enabled,0.1,Unifi support: syslog parser + port scan + SSH BF detection,collections crowdsecurity/apache2-logs,enabled,1.3,Parse Apache2 access and error logs,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/dropbear-logs,enabled,0.2,Parse dropbear logs,parsers crowdsecurity/http-logs,enabled,1.1,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/iptables-logs,enabled,0.3,Parse iptables drop logs,parsers crowdsecurity/nginx-logs,enabled,1.3,Parse nginx access and error logs,parsers crowdsecurity/sshd-logs,enabled,2.0,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/unifi-logs,enabled,0.1,,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers crowdsecurity/CVE-2022-26134,enabled,0.1,Detect CVE-2022-26134 exploits,scenarios crowdsecurity/CVE-2022-35914,enabled,0.1,Detect CVE-2022-35914 exploits,scenarios crowdsecurity/CVE-2022-37042,enabled,0.1,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/CVE-2022-40684,enabled,0.2,Detect cve-2022-40684 exploitation attempts,scenarios crowdsecurity/CVE-2022-41082,enabled,0.3,Detect CVE-2022-41082 exploits,scenarios crowdsecurity/CVE-2022-41697,enabled,0.1,Detect CVE-2022-41697 enumeration,scenarios crowdsecurity/CVE-2022-42889,enabled,0.2,Detect CVE-2022-42889 exploits (Text4Shell),scenarios crowdsecurity/CVE-2022-44877,enabled,0.2,Detect CVE-2022-44877 exploits,scenarios crowdsecurity/CVE-2022-46169,enabled,0.1,Detect CVE-2022-46169 brute forcing,scenarios crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.3,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.4,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.3,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/iptables-scan-multi_ports,enabled,0.1,ban IPs that are scanning us,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/nginx-req-limit-exceeded,enabled,0.1,Detects IPs which violate nginx's user set request limit.,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios ```

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* # filenames: - /var/log/nginx/*.log - ./tests/nginx/nginx.log #this is not a syslog log, indicate which kind of logs it is labels: type: nginx --- filenames: - /var/log/auth.log - /var/log/syslog labels: type: syslog --- source: journalctl journalctl_filter: - "_SYSTEMD_UNIT=ssh.service" labels: type: syslog --- filename: /var/log/apache2/*.log labels: type: apache2 --- source: syslog listen_addr: 0.0.0.0 listen_port: 4242 labels: type: unifi cat: '/etc/crowdsec/acquis.d/*': No such file or directory ```

Config show

```console $ cscli config show # Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /var/lib/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log/ - Log level : info - Log Media : file Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 - Acquisition Folder : /etc/crowdsec/acquis.d cscli: - Output : human - Hub Branch : - Hub Folder : /var/lib/crowdsec/hub Local API Server: - Listen URL : 127.0.0.1:8080 - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - 127.0.0.1 - ::1 - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000 ```

Prometheus metrics

```console $ cscli metrics # ERRO[18-01-2024 02:44:19] crowdsec - goroutine crowdsec/ShowPrometheus crashed : protobuf tag not enough fields in MetricFamily.state: ERRO[18-01-2024 02:44:19] please report this error to https://github.com/crowdsecurity/crowdsec/ ```

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

github-actions[bot] commented 7 months ago

@jstnhd1: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
LaurenceJJones commented 7 months ago

Do you have Prometheus enabled within CrowSec configuration could you share that part of the config? and are you running cscli metrics on the same server where crowdsec is running?

Edit: I missed the no other changes to config 👍🏻

jstnhd1 commented 7 months ago

yes, i am running on the same server.

when i try running the 127.0.0.1:6060, and the 8080 i get a ssl error. An error occurred during a connection to 127.0.0.1:6060. SSL received a record that exceeded the maximum permissible length

Edit: i missed the dashboard setup section, disregrard that second part.

blotus commented 7 months ago

Hello,

It looks like you are using the crowdsec package available in the official debian repositories.

I can reproduce the issue when running on trixie (but not on other debian versions), so I think it's a dependency issue on debian side when they build the crowdsec package. We'll report this to the maintainer of the package.

In the meantime, I'd recommend trying our own repositories (https://docs.crowdsec.net/docs/next/getting_started/install_crowdsec) so you can install a more recent version, this will fix the issue.

I'm going to close this as the problem is not really on our side.