search

crowdsecurity / hub

Main repository for crowdsec scenarios/parsers
https://hub.crowdsec.net
162 stars 151 forks source link

False positive http-crawl-non_statics Jellyseerr #1123

Open jalapeno1083 opened 2 months ago

jalapeno1083 commented 2 months ago

Describe the bug False positive when scrolling through Jellyseerr and loading a few different pages and scrolling down. This makes the client load many thumbnails.

edit: I just double checked. It only happens when I scroll down my request list domain.com/requests edit2: Ok it also happens, when browsing normally. Scrolling down the request list just gets you banned faster.

To Reproduce Install Jellyseerr and start browsing and scrolling. To get yourself banned even faster, fill your requests list and then scroll down the requests page domain.com/requests

Expected behavior

Using Jellyseerr normally like browsing, scrolling, loading thumbnails, and scrolling down the requests lists on Jellyseerr without getting banned.

Info about alert

cscli alerts inspect ################################################################################################ - ID : 154 - Date : 2024-09-29T14:30:09Z - Machine : localhost - Simulation : false - Remediation : true - Reason : crowdsecurity/http-crawl-non_statics - Events Count : 74 - Scope:Value : Ip:XXXXX - Country : XXXXX - AS : XXXXX - Begin : 2024-09-29 14:29:50.845089625 +0000 UTC - End : 2024-09-29 14:30:08.794147427 +0000 UTC - UUID : 76bc859b-7c5d-4eb7-b829-c1eb5a1c6594 - Context : +------------+---------------------+ | Key | Value | +------------+---------------------+ | method | GET | | status | 304 | | target_uri | /api/v1/request/585 | | target_uri | /api/v1/request/621 | | target_uri | /api/v1/request/642 | | target_uri | /api/v1/request/630 | | target_uri | /api/v1/request/633 | | target_uri | /api/v1/request/599 | | user_agent | - | +------------+---------------------+ - Events : - Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/585 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+ - Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/621 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+ - Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/642 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+ - Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/630 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+ - Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/633 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+ - Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/599 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+

Additional context

Collections in use COLLECTIONS ----------------------------------------------------------------------------------------------------------------- Name 📦 Status Version Local Path ----------------------------------------------------------------------------------------------------------------- crowdsecurity/base-http-scenarios ✔️ enabled 1.0 /etc/crowdsec/collections/base-http-scenarios.yaml crowdsecurity/http-cve ✔️ enabled 2.7 /etc/crowdsec/collections/http-cve.yaml crowdsecurity/iptables ✔️ enabled 0.2 /etc/crowdsec/collections/iptables.yaml crowdsecurity/linux ✔️ enabled 0.2 /etc/crowdsec/collections/linux.yaml crowdsecurity/nginx ✔️ enabled 0.2 /etc/crowdsec/collections/nginx.yaml crowdsecurity/sshd ✔️ enabled 0.5 /etc/crowdsec/collections/sshd.yaml crowdsecurity/traefik ✔️ enabled 0.1 /etc/crowdsec/collections/traefik.yaml crowdsecurity/whitelist-good-actors ✔️ enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml LePresidente/jellyfin ✔️ enabled 0.2 /etc/crowdsec/collections/jellyfin.yml LePresidente/jellyseerr ✔️ enabled 0.1 /etc/crowdsec/collections/jellyseerr.yml -----------------------------------------------------------------------------------------------------------------

Happy to provide any additional logs.

DanteMS commented 1 month ago

I'm using the following whitelist for Jellyseerr:

name: overseerr-jellyseerr-whitelist
description: "Whitelist events from Overseerr and Jellyseerr"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Overseerr/Jellyseerr whitelist"
  expression:
   - evt.Meta.http_status in ['200', '499'] && evt.Parsed.static_ressource == 'false' && evt.Meta.http_verb == 'GET' && evt.Meta.http_path matches '\\/api\\/v1\\/(movie|tv|request)\\/(\\d+)' # When browsing Movies, Series or Requests

Put it into /etc/crowdsec/config/parsers/s02-enrich. Maybe someone could merge it with the Jellyseerr and the Overseerr collections?

wacomoto commented 1 month ago

Thanks for the information. I've just recently encountered this issue.

My set up had been working with the acquisition template looking directly at /var/log/jellyseerr/overseerr-*.log. After getting banned (http status 403 then 200) I have updated the acquisition file per the hub example and added a whitelist for Jellyseerr. The regex expression above didn't work for me and I've found only whitlisting http_status 200 to have been sufficient (so far). My whitelist for Jellyseerr on traefik:

name: crowdsecurity/jellyseerr-whitelists
description: "Whitelist false positives from Jellyseerr api"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Whitelist false positive from Jellyseerr api"
  expression:
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/movie/'
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/tv/'
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/request/'