search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.88k stars 462 forks source link

High CPU on Pi Zero W #1944

Closed staticEndeavour closed 1 year ago

staticEndeavour commented 1 year ago

What happened? Crowdsec pegs the CPU on a Pi Zero W install. New to linux, new to issues, so forgive me if there's a process I'm missing. I installed Crowdsec on a Pi Zero W along with Pihole, the latter of which was giving me warnings about sustained load. I checked htop and found that every minute or so Crowdsec would run the CPU up to near 100% - specifically the following:

/usr/bin/crowdsec -c /etc/crowdsec/config.yaml -t

This lasts for about a minute and then the process repeats. Could be that it's too resource intensive for a Pi Zero, in which case that's fine, I'd have to remove as it could cause bottlenecks on the DNS resolution with pihole.

UFW is also installed and I've opened whatever ports the Crowdsec Readme mentions, but it's possible there

What did you expect to happen? Less resource usage

How can we reproduce it (as minimally and precisely as possible)? Install on a Pi Zero W

Anything else we need to know? Only applications I've installed are: UFW Pihole Crowdsec Tailscale

Crowdsec version

2022/12/23 14:10:40 version: v1.4.3-debian-pragmatic-f2528f3e2966d257905cca47fa1fa0e67cc2e2e8
2022/12/23 14:10:40 Codename: alphaga
2022/12/23 14:10:40 BuildDate: 2022-12-06_11:19:14
2022/12/23 14:10:40 GoVersion: 1.19.2
2022/12/23 14:10:40 Platform: linux
2022/12/23 14:10:40 Constraint_parser: >= 1.0, <= 2.0
2022/12/23 14:10:40 Constraint_scenario: >= 1.0, < 3.0
2022/12/23 14:10:40 Constraint_api: v1
2022/12/23 14:10:40 Constraint_acquis: >= 1.0, < 2.0

OS version

PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
github-actions[bot] commented 1 year ago

@staticEndeavour: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
github-actions[bot] commented 1 year ago

@staticEndeavour: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
LaurenceJJones commented 1 year ago

What logs are you consuming? Could you run

Cscli metrics

You don't need to open crowdsec ports to the Internet as by default listens on 127.0.0.1

staticEndeavour commented 1 year ago

Apologies for the delay. 


Acquisition Metrics:
╭─────────────────────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────╮
│                       Source                        │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │
├─────────────────────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┤
│ file:/var/log/auth.log                              │ 1.36k      │ 12           │ 1.35k          │ -                      │
│ file:/var/log/kern.log                              │ 12.34k     │ -            │ 12.34k         │ -                      │
│ file:/var/log/messages                              │ 12.34k     │ -            │ 12.34k         │ -                      │
│ file:/var/log/syslog                                │ 92.59k     │ -            │ 92.59k         │ -                      │
│ journalctl:journalctl-_SYSTEMD_UNIT=apache2.service │ 3          │ -            │ 3              │ -                      │
╰─────────────────────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────╯

Parser Metrics:
╭──────────────────────────────────┬─────────┬─────────┬──────────╮
│             Parsers              │  Hits   │ Parsed  │ Unparsed │
├──────────────────────────────────┼─────────┼─────────┼──────────┤
│ child-crowdsecurity/apache2-logs │ 6       │ -       │ 6        │
│ child-crowdsecurity/sshd-logs    │ 616     │ 12      │ 604      │
│ child-crowdsecurity/syslog-logs  │ 118.62k │ 118.62k │ -        │
│ crowdsecurity/apache2-logs       │ 3       │ -       │ 3        │
│ crowdsecurity/dateparse-enrich   │ 12      │ 12      │ -        │
│ crowdsecurity/geoip-enrich       │ 12      │ 12      │ -        │
│ crowdsecurity/non-syslog         │ 3       │ 3       │ -        │
│ crowdsecurity/sshd-logs          │ 69      │ 12      │ 57       │
│ crowdsecurity/syslog-logs        │ 118.62k │ 118.62k │ -        │
│ crowdsecurity/whitelists         │ 12      │ 12      │ -        │
╰──────────────────────────────────┴─────────┴─────────┴──────────╯

Local Api Metrics:
╭────────────────────┬────────┬───────╮
│       Route        │ Method │ Hits  │
├────────────────────┼────────┼───────┤
│ /v1/heartbeat      │ GET    │ 11249 │
│ /v1/watchers/login │ POST   │ 192   │
╰────────────────────┴────────┴───────╯

Local Api Machines Metrics:
╭──────────────────────────────────────────────────┬───────────────┬────────┬───────╮
│                     Machine                      │     Route     │ Method │ Hits  │
├──────────────────────────────────────────────────┼───────────────┼────────┼───────┤
│ 44e7febc1e094ae8815e1ede19ce2e37wbfbfKq2jVCthvP0 │ /v1/heartbeat │ GET    │ 11249 │
╰──────────────────────────────────────────────────┴───────────────┴────────┴───────╯

Local Api Decisions:
╭────────────────────────────────────────────┬────────┬────────┬───────╮
│                   Reason                   │ Origin │ Action │ Count │
├────────────────────────────────────────────┼────────┼────────┼───────┤
│ crowdsecurity/grafana-cve-2021-43798       │ CAPI   │ ban    │ 1     │
│ crowdsecurity/http-open-proxy              │ CAPI   │ ban    │ 56    │
│ crowdsecurity/jira_cve-2021-26086          │ CAPI   │ ban    │ 20    │
│ crowdsecurity/ssh-bf                       │ CAPI   │ ban    │ 6209  │
│ crowdsecurity/thinkphp-cve-2018-20062      │ CAPI   │ ban    │ 9     │
│ crowdsecurity/CVE-2022-41082               │ CAPI   │ ban    │ 1     │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ CAPI   │ ban    │ 9     │
│ crowdsecurity/f5-big-ip-cve-2020-5902      │ CAPI   │ ban    │ 3     │
│ ltsich/http-w00tw00t                       │ CAPI   │ ban    │ 1     │
│ crowdsecurity/http-cve-2021-41773          │ CAPI   │ ban    │ 29    │
│ crowdsecurity/spring4shell_cve-2022-22965  │ CAPI   │ ban    │ 1     │
│ crowdsecurity/fortinet-cve-2018-13379      │ CAPI   │ ban    │ 10    │
│ crowdsecurity/http-backdoors-attempts      │ CAPI   │ ban    │ 44    │
│ crowdsecurity/http-bad-user-agent          │ CAPI   │ ban    │ 1121  │
│ crowdsecurity/http-crawl-non_statics       │ CAPI   │ ban    │ 214   │
│ crowdsecurity/http-generic-bf              │ CAPI   │ ban    │ 4     │
│ crowdsecurity/http-probing                 │ CAPI   │ ban    │ 1121  │
│ crowdsecurity/ssh-slow-bf                  │ CAPI   │ ban    │ 2309  │
│ crowdsecurity/CVE-2022-35914               │ CAPI   │ ban    │ 1     │
│ crowdsecurity/http-path-traversal-probing  │ CAPI   │ ban    │ 45    │
│ crowdsecurity/http-sensitive-files         │ CAPI   │ ban    │ 42    │
╰────────────────────────────────────────────┴────────┴────────┴───────╯
LaurenceJJones commented 1 year ago

The http collection has crowdsecurity/http-bad-user-agent which is very CPU intensive especially for a 1 core ARM CPU I would recommend removing that.