search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.87k stars 459 forks source link

restart loop "Error while installing 'crowdsecurity/traefik'" #1946

Closed jonwilliams84 closed 1 year ago

jonwilliams84 commented 1 year ago

What happened?

Was running v1.4.2 and decided to update to v1.4.3. Now the container constantly restarts with the following error:

time="24-12-2022 10:09:47 AM" level=warning msg="crowdsecurity/traefik : overwrite"
time="24-12-2022 10:09:47 AM" level=fatal msg="Error while installing 'crowdsecurity/traefik': while enabling crowdsecurity/traefik: while installing crowdsecurity/base-http-scenarios: while installing crowdsecurity/http-cve: crowdsecurity/http-cve is tainted, won't enable unless --force"

I have dropped back to v1.4.2 and it works fine.

What did you expect to happen?

Update to v1.4.3 without this issue.

How can we reproduce it (as minimally and precisely as possible)?

update from v1.4.2 to 1.4.3 with 'crowdsecurity/traefik' and these collections:

COLLECTIONS
--------------------------------------------------------------------------------------------------------------------------
 Name                                  📦 Status           Version   Local Path
--------------------------------------------------------------------------------------------------------------------------
 LePresidente/emby                     ✔️ enabled           0.1       /etc/crowdsec/collections/emby.yml
 crowdsecurity/base-http-scenarios     ⚠️ enabled,tainted   0.6       /etc/crowdsec/collections/base-http-scenarios.yaml
 crowdsecurity/http-cve                ⚠️ enabled,tainted   1.6       /etc/crowdsec/collections/http-cve.yaml
 crowdsecurity/linux                   ✔️ enabled           0.2       /etc/crowdsec/collections/linux.yaml
 crowdsecurity/sshd                    ✔️ enabled           0.2       /etc/crowdsec/collections/sshd.yaml
 crowdsecurity/traefik                 ✔️ enabled           0.1       /etc/crowdsec/collections/traefik.yaml
 crowdsecurity/whitelist-good-actors   ✔️ enabled           0.1       /etc/crowdsec/collections/whitelist-good-actors.yaml

Anything else we need to know?

No response

Crowdsec version

v1.4.3

OS version

NAME="Ubuntu" VERSION="18.04.6 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.6 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic

5.4.0-135-generic #152~18.04.2-Ubuntu SMP Tue Nov 29 08:23:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Enabled collections and parsers

LePresidente/emby,enabled,0.1,Emby support : parser and brute-force detection,collections crowdsecurity/base-http-scenarios,"enabled,tainted",0.6,http common : scanners detection,collections crowdsecurity/http-cve,"enabled,tainted",1.6,,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections crowdsecurity/traefik,enabled,0.1,traefik support: parser and generic http scenarios,collections crowdsecurity/whitelist-good-actors,enabled,0.1,Good actors whitelists,collections LePresidente/emby-logs,enabled,0.2,Parse emby logs,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/docker-logs,enabled,0.1,docker json logs parser,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,"enabled,tainted",?,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/sshd-logs,enabled,2.0,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/traefik-logs,enabled,0.5,Parse Traefik access logs,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers LePresidente/emby-bf,enabled,0.1,Detect emby bruteforce,scenarios crowdsecurity/CVE-2022-26134,enabled,0.1,Detect CVE-2022-26134 exploits,scenarios crowdsecurity/CVE-2022-35914,enabled,0.1,Detect CVE-2022-35914 exploits,scenarios crowdsecurity/CVE-2022-37042,enabled,0.1,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/CVE-2022-40684,enabled,0.2,Detect cve-2022-40684 exploitation attempts,scenarios crowdsecurity/CVE-2022-41082,enabled,0.3,Detect CVE-2022-41082 exploits,scenarios crowdsecurity/CVE-2022-42889,enabled,0.2,Detect CVE-2022-42889 exploits (Text4Shell),scenarios crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,"enabled,tainted",?,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.3,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.2,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.3,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios crowdsecurity/cdn-whitelist,enabled,0.4,Whitelist CDN providers,postoverflows crowdsecurity/rdns,enabled,0.2,Lookup the DNS associated to the source IP only for overflows,postoverflows crowdsecurity/seo-bots-whitelist,enabled,0.4,Whitelist good search engine crawlers,postoverflows

Acquisition config

filenames:

Config show

Global:

Prometheus metrics

Acquisition Metrics: +-----------------------------------+------------+--------------+----------------+------------------------+ | Source | Lines read | Lines parsed | Lines unparsed | Lines poured to bucket | +-----------------------------------+------------+--------------+----------------+------------------------+ | file:/var/log/emby/embyserver.txt | 160 | - | 160 | - | | file:/var/log/traefik/access.log | 116 | 116 | - | - | +-----------------------------------+------------+--------------+----------------+------------------------+

Parser Metrics: +----------------------------------+------+--------+----------+ | Parsers | Hits | Parsed | Unparsed | +----------------------------------+------+--------+----------+ | LePresidente/emby-logs | 160 | - | 160 | | child-LePresidente/emby-logs | 160 | - | 160 | | child-crowdsecurity/http-logs | 348 | 232 | 116 | | child-crowdsecurity/traefik-logs | 116 | 116 | - | | crowdsecurity/dateparse-enrich | 116 | 116 | - | | crowdsecurity/geoip-enrich | 116 | 116 | - | | crowdsecurity/http-logs | 116 | 116 | - | | crowdsecurity/non-syslog | 276 | 276 | - | | crowdsecurity/traefik-logs | 116 | 116 | - | | crowdsecurity/whitelists | 116 | 116 | - | +----------------------------------+------+--------+----------+

Local Api Metrics: +--------------------+--------+------+ | Route | Method | Hits | +--------------------+--------+------+ | /v1/decisions | GET | 80 | | /v1/heartbeat | GET | 13 | | /v1/watchers/login | POST | 2 | +--------------------+--------+------+

Local Api Machines Metrics: +-----------+---------------+--------+------+ | Machine | Route | Method | Hits | +-----------+---------------+--------+------+ | localhost | /v1/heartbeat | GET | 13 | +-----------+---------------+--------+------+

Local Api Bouncers Metrics: +-----------------+---------------+--------+------+ | Bouncer | Route | Method | Hits | +-----------------+---------------+--------+------+ | bouncer-traefik | /v1/decisions | GET | 80 | +-----------------+---------------+--------+------+

Local Api Bouncers Decisions: +-----------------+---------------+-------------------+ | Bouncer | Empty answers | Non-empty answers | +-----------------+---------------+-------------------+ | bouncer-traefik | 80 | 0 | +-----------------+---------------+-------------------+

Local Api Decisions: +--------------------------------------------+--------+--------+-------+ | Reason | Origin | Action | Count | +--------------------------------------------+--------+--------+-------+ | crowdsecurity/http-generic-bf | CAPI | ban | 13 | | crowdsecurity/http-open-proxy | CAPI | ban | 177 | | crowdsecurity/jira_cve-2021-26086 | CAPI | ban | 116 | | crowdsecurity/ssh-slow-bf | CAPI | ban | 1114 | | crowdsecurity/CVE-2022-41082 | CAPI | ban | 1 | | crowdsecurity/http-backdoors-attempts | CAPI | ban | 140 | | crowdsecurity/http-crawl-non_statics | CAPI | ban | 811 | | crowdsecurity/vmware-cve-2022-22954 | CAPI | ban | 4 | | crowdsecurity/f5-big-ip-cve-2020-5902 | CAPI | ban | 3 | | crowdsecurity/http-bad-user-agent | CAPI | ban | 4855 | | crowdsecurity/http-path-traversal-probing | CAPI | ban | 272 | | crowdsecurity/thinkphp-cve-2018-20062 | CAPI | ban | 20 | | ltsich/http-w00tw00t | CAPI | ban | 4 | | crowdsecurity/CVE-2022-35914 | CAPI | ban | 2 | | crowdsecurity/fortinet-cve-2018-13379 | CAPI | ban | 44 | | crowdsecurity/http-cve-2021-42013 | CAPI | ban | 1 | | crowdsecurity/http-cve-2021-41773 | CAPI | ban | 72 | | crowdsecurity/http-probing | CAPI | ban | 3442 | | crowdsecurity/http-sensitive-files | CAPI | ban | 123 | | crowdsecurity/spring4shell_cve-2022-22965 | CAPI | ban | 2 | | crowdsecurity/ssh-bf | CAPI | ban | 5422 | | crowdsecurity/CVE-2022-37042 | CAPI | ban | 3 | | crowdsecurity/apache_log4j2_cve-2021-44228 | CAPI | ban | 29 | | crowdsecurity/grafana-cve-2021-43798 | CAPI | ban | 3 | +--------------------------------------------+--------+--------+-------+

Local Api Alerts: +--------------------------------------+-------+ | Reason | Count | +--------------------------------------+-------+ | crowdsecurity/http-crawl-non_statics | 7 | | crowdsecurity/http-probing | 4 | +--------------------------------------+-------+

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

github-actions[bot] commented 1 year ago

@jonwilliams84: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
jonwilliams84 commented 1 year ago

Ok...I think I have managed to resolve it by forcefully upgrading the collections that were tainted.

cscli collections upgrade crowdsecurity/base-http-scenarios --force
cscli collections upgrade crowdsecurity/http-cve --force
cscli collections upgrade crowdsecurity/traefik --force
LaurenceJJones commented 1 year ago

Issue resolved in docker version 1.4.4-rc2 linked to #1916

There was an error in 1.4.3 startup script, which would need no tainted files (you edited the scenarios, most likely due to a false positive), however, using force will revert any changes you made