search

crowdsecurity / crowdsec

CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
https://crowdsec.net
MIT License
8.98k stars 466 forks source link

Telegram Notifications do not work #3292

Closed l4rm4nd closed 1 week ago

l4rm4nd commented 1 week ago

What happened?

I've configured Telegram notifications for Crowdsec + Traefik. Both run as Docker container.

If I test the notifications via cscli, I successfully receive a test notification in Telegram:

docker exec crowdsec cscli notifications test http_telegram

Command results are:

time="2024-10-22T02:01:46Z" level=debug msg="starting plugin" args="[/usr/local/lib/crowdsec/plugins/notification-http]" path=/usr/local/lib/crowdsec/plugins/notification-http
time="2024-10-22T02:01:46Z" level=debug msg="plugin started" path=/usr/local/lib/crowdsec/plugins/notification-http pid=495
time="2024-10-22T02:01:46Z" level=debug msg="waiting for RPC address" path=/usr/local/lib/crowdsec/plugins/notification-http
time="2024-10-22T02:01:46Z" level=debug msg="using plugin" version=1
time="2024-10-22T02:01:46Z" level=trace msg="waiting for stdio data"
level=info msg="registered plugin http_telegram"
level=info msg="pluginTomb dying"
time="2024-10-22T02:01:46Z" level=info msg="received signal for http_telegram config" @module=http-plugin
level=info msg="killing all plugins"
time="2024-10-22T02:01:46Z" level=debug msg="received EOF, stopping recv loop" err="rpc error: code = Unavailable desc = error reading from server: read unix @->/tmp/plugin3001699612: read: connection reset by peer"
time="2024-10-22T02:01:46Z" level=info msg="plugin process exited" path=/usr/local/lib/crowdsec/plugins/notification-http pid=495
time="2024-10-22T02:01:46Z" level=debug msg="plugin exited"

However, for ban actions, I do not receive any notifications. The log show this error:

crowdsec  | time="2024-10-22T01:45:14Z" level=info msg="Ip <MASKED-IP> performed 'crowdsecurity/http-probing' (11 events over 276.18955ms) at 2024-10-22 01:45:14.240202071 +0000 UTC"
crowdsec  | time="2024-10-22T01:45:14Z" level=warning msg="Cannot send alert to Plugin channel (try: 0)"

What did you expect to happen?

I receive a Telegram notification once a ban is issued.

How can we reproduce it (as minimally and precisely as possible)?

Configure Telegram notifications by editing the /etc/profiles.yaml and /etc/notifications/http.yaml.

Basically, according to https://docs.crowdsec.net/docs/next/notification_plugins/telegram/

Anything else we need to know?

IP banning works; crowdsec logs look fine but no telegram notifications

Crowdsec version

```console version: v1.6.3-4851945a Codename: alphaga BuildDate: 2024-09-12_09:39:08 GoVersion: 1.22.6 Platform: docker libre2: C++ User-Agent: crowdsec/v1.6.3-4851945a-docker Constraint_parser: >= 1.0, <= 3.0 Constraint_scenario: >= 1.0, <= 3.0 Constraint_api: v1 Constraint_acquis: >= 1.0, < 2.0 ```

OS version

```console PRETTY_NAME="Ubuntu 22.04.5 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.5 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy Linux ubuntu-docker 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux ```

Enabled collections and parsers

```console name,status,version,description,type crowdsecurity/cri-logs,enabled,0.1,CRI logging format parser,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/docker-logs,enabled,0.1,docker json logs parser,parsers crowdsecurity/geoip-enrich,enabled,0.5,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,1.2,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/sshd-logs,enabled,2.8,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/traefik-logs,enabled,0.9,Parse Traefik access logs,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.6,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/CVE-2017-9841,enabled,0.2,Detect CVE-2017-9841 exploits,scenarios crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios crowdsecurity/CVE-2022-26134,enabled,0.2,Detect CVE-2022-26134 exploits,scenarios crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios crowdsecurity/CVE-2022-44877,enabled,0.3,Detect CVE-2022-44877 exploits,scenarios crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios crowdsecurity/CVE-2023-22518,enabled,0.2,Detect CVE-2023-22518 exploits,scenarios crowdsecurity/CVE-2023-49103,enabled,0.3,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios crowdsecurity/CVE-2024-38475,enabled,0.1,Detect CVE-2024-38475 exploitation attempts,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.2,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.3,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.2,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-admin-interface-probing,enabled,0.4,Detect generic HTTP admin interface probing,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.6,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,1.2,Detect usage of bad User Agent,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.7,Detect aggressive crawl on non static resources,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.2,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.2,cve-2021-42013,scenarios crowdsecurity/http-cve-probing,enabled,0.2,Detect generic HTTP cve probing,scenarios crowdsecurity/http-generic-bf,enabled,0.6,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.5,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.4,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.4,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.4,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.4,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-wordpress-scan,enabled,0.2,Detect WordPress scan: vuln hunting,scenarios crowdsecurity/http-xss-probing,enabled,0.4,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.3,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/netgear_rce,enabled,0.3,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.3,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios crowdsecurity/ssh-cve-2024-6387,enabled,0.2,Detect exploitation attempt of CVE-2024-6387,scenarios crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.6,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.2,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.2,detect w00tw00t,scenarios crowdsecurity/bf_base,enabled,0.1,,contexts crowdsecurity/http_base,enabled,0.2,,contexts crowdsecurity/base-http-scenarios,enabled,1.0,http common : scanners detection,collections crowdsecurity/http-cve,enabled,2.7,Detect CVE exploitation in http logs,collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/sshd,enabled,0.5,sshd support : parser and brute-force detection,collections crowdsecurity/traefik,enabled,0.1,traefik support: parser and generic http scenarios,collections ```

Acquisition config

```` filenames: - /var/log/auth.log labels: type: syslog --- filenames: - /var/log/traefik/*.log labels: type: traefik ````

Config show

```console Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log - Log level : info - Log Media : stdout Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 - Acquisition Folder : /etc/crowdsec/acquis.d cscli: - Output : human - Hub Branch : API Client: - URL : http://0.0.0.0:8080/ - Login : localhost - Credentials File : /etc/crowdsec/local_api_credentials.yaml Local API Server: - Listen URL : 0.0.0.0:8080 - Listen Socket : - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - 127.0.0.1 - ::1 - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000 ```

Prometheus metrics

No response

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

profiles.yaml

name: default_ip_remediation
#debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
 - type: ban
   duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
# notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
#   - http_default   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success: break
---
name: default_range_remediation
#debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Range"
decisions:
 - type: ban
   duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
 - http_telegram  # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success: break

http.yaml

type: http          # Don't change
name: http_telegram  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body

# Replace XXXXXXXXX with your Telegram chat ID
format: |
  {
   "chat_id": "123456",
   "text": "
     {{range . -}}
     {{$alert := . -}}
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
    }
  }

url: https://api.telegram.org/bot<MASKED-API-TOKEN>/sendMessage # Replace <TELEGRAM_API_KEY> with your API key

method: POST
headers:
  Content-Type: "application/json"
github-actions[bot] commented 1 week ago

@l4rm4nd: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.
l4rm4nd commented 1 week ago

I've missed enabling the notifications for default_ip_remediation. Therefore, no notifications were sent out.

My fail, sorry.

Works now as intended.