search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
9.16k stars 472 forks source link

crowdsec without any blocklists blocks github web hooks #2877

Closed Morriz closed 8 months ago

Morriz commented 9 months ago

What happened?

I receive github webhooks, but those get a 403 from crowdsec, even after removing the db and starting fresh without any blocklists.

What did you expect to happen?

No 403?

How can we reproduce it (as minimally and precisely as possible)?

add a github web hook to a project and trigger delivery

Anything else we need to know?

No response

Crowdsec version

```console $ cscli version 2024/03/07 10:38:32 version: v1.6.0-4b8e6cd7 2024/03/07 10:38:32 Codename: alphaga 2024/03/07 10:38:32 BuildDate: 2024-01-24_15:04:15 2024/03/07 10:38:32 GoVersion: 1.21.6 2024/03/07 10:38:32 Platform: docker 2024/03/07 10:38:32 libre2: C++ 2024/03/07 10:38:32 Constraint_parser: >= 1.0, <= 3.0 2024/03/07 10:38:32 Constraint_scenario: >= 1.0, <= 3.0 2024/03/07 10:38:32 Constraint_api: v1 2024/03/07 10:38:32 Constraint_acquis: >= 1.0, < 2.0 ```

OS version

```console # On Linux: $ cat /etc/os-release NAME="Alpine Linux" ID=alpine VERSION_ID=3.19.0 PRETTY_NAME="Alpine Linux v3.19" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues" $ uname -a Linux 755c033d5c7f 6.1.0-rpi8-rpi-2712 #1 SMP PREEMPT Debian 1:6.1.73-1+rpt1 (2024-01-25) aarch64 Linux ```

Enabled collections and parsers

```console $ cscli hub list -o raw name,status,version,description,type crowdsecurity/appsec-logs,enabled,0.5,Parse Appsec events,parsers crowdsecurity/cri-logs,enabled,0.1,CRI logging format parser,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/docker-logs,enabled,0.1,docker json logs parser,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,1.2,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/sshd-logs,enabled,2.3,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/traefik-logs,enabled,0.9,Parse Traefik access logs,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers crowdsecurity/cdn-whitelist,enabled,0.4,Whitelist CDN providers,postoverflows crowdsecurity/rdns,enabled,0.3,Lookup the DNS associated to the source IP only for overflows,postoverflows crowdsecurity/seo-bots-whitelist,enabled,0.4,Whitelist good search engine crawlers,postoverflows crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.5,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/appsec-vpatch,enabled,0.5,Identify attacks flagged by CrowdSec AppSec,scenarios crowdsecurity/CVE-2017-9841,enabled,0.2,Detect CVE-2017-9841 exploits,scenarios crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios crowdsecurity/CVE-2022-26134,enabled,0.2,Detect CVE-2022-26134 exploits,scenarios crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios crowdsecurity/CVE-2022-44877,enabled,0.3,Detect CVE-2022-44877 exploits,scenarios crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios crowdsecurity/CVE-2023-22518,enabled,0.2,Detect CVE-2023-22518 exploits,scenarios crowdsecurity/CVE-2023-49103,enabled,0.3,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.2,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.3,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.2,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-admin-interface-probing,enabled,0.3,Detect generic HTTP admin interface probing,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.5,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,1.1,Detect usage of bad User Agent,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.7,Detect aggressive crawl on non static resources,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.2,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.2,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.6,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.5,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.3,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.3,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.3,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.3,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.3,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.2,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/netgear_rce,enabled,0.3,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.3,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.4,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.2,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.2,detect w00tw00t,scenarios crowdsecurity/bf_base,enabled,0.1,,contexts crowdsecurity/http_base,enabled,0.2,,contexts crowdsecurity/virtual-patching,enabled,0.3,,appsec-configs crowdsecurity/base-config,enabled,0.1,,appsec-rules crowdsecurity/vpatch-connectwise-auth-bypass,enabled,0.2,Detect exploitation of auth bypass in ConnectWise ScreenConnect,appsec-rules crowdsecurity/vpatch-CVE-2017-9841,enabled,0.3,PHPUnit RCE (CVE-2017-9841),appsec-rules crowdsecurity/vpatch-CVE-2018-1000861,enabled,0.1,Jenkins - RCE (CVE-2018-1000861),appsec-rules crowdsecurity/vpatch-CVE-2018-10562,enabled,0.2,Dasan GPON RCE (CVE-2018-10562),appsec-rules crowdsecurity/vpatch-CVE-2019-1003030,enabled,0.1,Jenkins - RCE (CVE-2019-1003030),appsec-rules crowdsecurity/vpatch-CVE-2019-12989,enabled,0.3,Citrix SQLi (CVE-2019-12989),appsec-rules crowdsecurity/vpatch-CVE-2020-11738,enabled,0.6,Wordpress Snap Creek Duplicator - Path Traversal (CVE-2020-11738),appsec-rules crowdsecurity/vpatch-CVE-2020-17496,enabled,0.1,vBulletin RCE (CVE-2020-17496),appsec-rules crowdsecurity/vpatch-CVE-2021-22941,enabled,0.3,Citrix RCE (CVE-2021-22941),appsec-rules crowdsecurity/vpatch-CVE-2021-3129,enabled,0.4,Laravel with Ignition Debug Mode RCE (CVE-2021-3129),appsec-rules crowdsecurity/vpatch-CVE-2022-22965,enabled,0.2,Spring4Shell - RCE (CVE-2022-22965),appsec-rules crowdsecurity/vpatch-CVE-2022-27926,enabled,0.4,Zimbra Collaboration XSS (CVE-2022-27926),appsec-rules crowdsecurity/vpatch-CVE-2022-35914,enabled,0.5,GLPI RCE (CVE-2022-35914),appsec-rules crowdsecurity/vpatch-CVE-2022-44877,enabled,0.2,CentOS Web Panel 7 RCE (CVE-2022-44877),appsec-rules crowdsecurity/vpatch-CVE-2022-46169,enabled,0.5,Cacti RCE (CVE-2022-46169),appsec-rules crowdsecurity/vpatch-CVE-2023-1389,enabled,0.1,TP-Link Archer AX21 - RCE (CVE-2023-1389),appsec-rules crowdsecurity/vpatch-CVE-2023-20198,enabled,0.6,CISCO IOS XE Account Creation (CVE-2023-20198),appsec-rules crowdsecurity/vpatch-CVE-2023-22515,enabled,0.4,Atlassian Confluence Privesc (CVE-2023-22515),appsec-rules crowdsecurity/vpatch-CVE-2023-22527,enabled,0.2,RCE using SSTI in Confluence (CVE-2023-22527),appsec-rules crowdsecurity/vpatch-CVE-2023-23752,enabled,0.1,Joomla! Webservice - Password Disclosure (CVE-2023-23752),appsec-rules crowdsecurity/vpatch-CVE-2023-24489,enabled,0.2,Citrix ShareFile RCE (CVE-2023-24489),appsec-rules crowdsecurity/vpatch-CVE-2023-28121,enabled,0.1,WooCommerce auth bypass (CVE-2023-28121),appsec-rules crowdsecurity/vpatch-CVE-2023-33617,enabled,0.4,Atlassian Confluence Privesc (CVE-2023-33617),appsec-rules crowdsecurity/vpatch-CVE-2023-34362,enabled,0.6,MOVEit Transfer RCE (CVE-2023-34362),appsec-rules crowdsecurity/vpatch-CVE-2023-35078,enabled,0.1,MobileIron Core Remote Unauthenticated API Access (CVE-2023-35078),appsec-rules crowdsecurity/vpatch-CVE-2023-35082,enabled,0.2,MobileIron Core Remote Unauthenticated API Access (CVE-2023-35082),appsec-rules crowdsecurity/vpatch-CVE-2023-3519,enabled,0.3,Citrix RCE (CVE-2023-3519),appsec-rules crowdsecurity/vpatch-CVE-2023-38205,enabled,0.3,Adobe ColdFusion Access Control Bypass (CVE-2023-38205),appsec-rules crowdsecurity/vpatch-CVE-2023-40044,enabled,0.3,WS_FTP .NET deserialize RCE (CVE-2023-40044),appsec-rules crowdsecurity/vpatch-CVE-2023-42793,enabled,0.3,JetBrains Teamcity Auth Bypass (CVE-2023-42793),appsec-rules crowdsecurity/vpatch-CVE-2023-46805,enabled,0.4,Ivanti Connect Auth Bypass (CVE-2023-46805),appsec-rules crowdsecurity/vpatch-CVE-2023-49070,enabled,0.1,Apache OFBiz - RCE (CVE-2023-49070),appsec-rules crowdsecurity/vpatch-CVE-2023-50164,enabled,0.6,Apache Struts2 Path Traversal (CVE-2023-50164),appsec-rules crowdsecurity/vpatch-CVE-2023-6553,enabled,0.1,Backup Migration plugin for WordPress RCE (CVE-2023-6553),appsec-rules crowdsecurity/vpatch-CVE-2023-7028,enabled,0.2,Gitlab Password Reset Account Takeover (CVE-2023-7028),appsec-rules crowdsecurity/vpatch-CVE-2024-23897,enabled,0.3,Jenkins CLI RCE (CVE-2023-50164),appsec-rules crowdsecurity/vpatch-env-access,enabled,0.1,Detect access to .env files,appsec-rules crowdsecurity/vpatch-laravel-debug-mode,enabled,0.3,Detect bots exploiting laravel debug mode,appsec-rules crowdsecurity/vpatch-symfony-profiler,enabled,0.1,Detect abuse of symfony profiler,appsec-rules crowdsecurity/appsec-virtual-patching,enabled,1.9,"a generic virtual patching collection, suitable for most web servers.",collections crowdsecurity/base-http-scenarios,enabled,0.8,http common : scanners detection,collections crowdsecurity/http-cve,enabled,2.6,Detect CVE exploitation in http logs,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/sshd,enabled,0.3,sshd support : parser and brute-force detection,collections crowdsecurity/traefik,enabled,0.1,traefik support: parser and generic http scenarios,collections crowdsecurity/whitelist-good-actors,enabled,0.1,Good actors whitelists,collections ```

Acquisition config

---
filenames:
  - /var/log/auth.log
  - /var/log/syslog
labels:
  type: syslog
---
filenames:
  - /var/log/traefik/access.log
labels:
  type: traefik

Config show

```console $ cscli config show # paste output here ```

Prometheus metrics

```console $ cscli metrics Acquisition Metrics: ╭──────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────╮ │ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ ├──────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┤ │ file:/var/log/traefik/access.log │ 12 │ 12 │ - │ - │ ╰──────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────╯ Parser Metrics: ╭──────────────────────────────────┬──────┬────────┬──────────╮ │ Parsers │ Hits │ Parsed │ Unparsed │ ├──────────────────────────────────┼──────┼────────┼──────────┤ │ child-crowdsecurity/http-logs │ 36 │ 30 │ 6 │ │ child-crowdsecurity/traefik-logs │ 12 │ 12 │ - │ │ crowdsecurity/dateparse-enrich │ 12 │ 12 │ - │ │ crowdsecurity/geoip-enrich │ 12 │ 12 │ - │ │ crowdsecurity/http-logs │ 12 │ 12 │ - │ │ crowdsecurity/non-syslog │ 12 │ 12 │ - │ │ crowdsecurity/traefik-logs │ 12 │ 12 │ - │ │ crowdsecurity/whitelists │ 12 │ 12 │ - │ ╰──────────────────────────────────┴──────┴────────┴──────────╯ Local API Metrics: ╭────────────────────┬────────┬──────╮ │ Route │ Method │ Hits │ ├────────────────────┼────────┼──────┤ │ /v1/heartbeat │ GET │ 5 │ │ /v1/watchers/login │ POST │ 1 │ ╰────────────────────┴────────┴──────╯ Local API Machines Metrics: ╭──────────┬───────────────┬────────┬──────╮ │ Machine │ Route │ Method │ Hits │ ├──────────┼───────────────┼────────┼──────┤ │ crowdsec │ /v1/heartbeat │ GET │ 5 │ ╰──────────┴───────────────┴────────┴──────╯ ```

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

github-actions[bot] commented 9 months ago

@Morriz: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
Morriz commented 9 months ago

other requests get handled just fine

LaurenceJJones commented 9 months ago

What happened?

I receive github webhooks, but those get a 403 from crowdsec, even after removing the db and starting fresh without any blocklists.

What did you expect to happen?

No 403?

How can we reproduce it (as minimally and precisely as possible)?

add a github web hook to a project and trigger delivery

Anything else we need to know?

No response

Crowdsec version

OS version

Enabled collections and parsers

Acquisition config

---
filenames:
  - /var/log/auth.log
  - /var/log/syslog
labels:
  type: syslog
---
filenames:
  - /var/log/traefik/access.log
labels:
  type: traefik

Config show

Prometheus metrics

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

Well there must of been a decision against the IP address, and even after clearing things there might be a time lapse between when your remediation component fetches the deleted IP addresses. You should consult your bouncer documentation to ensure which mode you are running in if its not LIVE which is the common naming scheme then this would be intended behaviour until it repulls. (Also there might be a intended bug/issue instead of deleting it via cscli decsions delete and deleting the database its impossible for crowdsec to know what state the bouncer is currently in, so it would of been more appropriate to delete the decisions or all via cscli decisions delete --all)

We had it in the past when a user wanted to do this, the issue we have is github uses azure serverless infra and in doing so can be mixed in with others that use azure serverless for malicious behaviors. the user in the end, use a VPN from github action to their infrastructure which is recommended way around this issue as clearly the ip address is malicious but for the seconds you have it contacting your infra it is not, unless you want to whitelist the whole of azure ASN's

https://medium.com/@mahmutali.mas/access-private-server-with-github-actions-vpn-connection-cf20313abdd5