search

maxlerebourg / crowdsec-bouncer-traefik-plugin

Traefik plugin for Crowdsec - WAF and IP protection
Apache License 2.0
260 stars 14 forks source link

[BUG/Question] Stream mode - Crowdsec LAPI KEY - UpdateIntervalSeconds #107

Closed guillaumekoto closed 1 year ago

guillaumekoto commented 1 year ago

Hi,

I have few questions about "stream" mode. Since I enabled it on traefik routers, the "Last API Pull" date didn't change anymore for concerned bouncers (cscli bouncer list). Only one bouncer (configured in stream mode) have an "Last API Pull" date which change every 60s (interval configured by default in plugin to update the cache). It's looks like that only one API KEY is used to update local cache. All others bouncer key aren't not used. I think it's normal, because it's not needed to update every 60s the cache for every bouncers. Cache is shared.

Questions:

In documentation you advise to define one API key per router.

mathieuHa commented 1 year ago

It's looks like that only one API KEY is used to update local cache. All others bouncer key aren't not used. I think it's normal, because it's not needed to update every 60s the cache for every bouncers. Cache is shared.

Yes, you are right, the cache is shared and every instance, will use locks to know if an update is already occuring.

How the plugin choose which bouncer API KEY is used to update the cache? (The first routers used after traefik starting?)

It is based on locks / timer schedule. Their is no way to predict which one will fetch data

Is it possible to define one crowdsec middleware in stream mode for all routers? In this case we don't create multiple useless API KEY.

No it is not, and it is better to use just one.

In this context (multiple routers in stream mode), how the "UpdateIntervalSeconds" could work's?

I'm not sure I follow the question, but since only one instance can fetch the stream, this value is used exclusively to define the period.

What is the goal of using redis to store the cache? (performance compared to filesystem? Persistence with huge cache size?)

The purpose is to export the caching mecanism out of Traefik.
I believe each component should serve it's purpose and having the caching mecanism outside of Traefik could in theorie be better for scalability.
However I don't have benchmarks showing that, and depending on your load you might not need/want to use redis for this.

I hope I awnsered the questions, if not, feel free.
I will update the examples and the readme, to make clearer, In stream mode, you only need one instance.

Best,

guillaumekoto commented 1 year ago

Thank you for the reply. I deployed traefik and crowdsec-plugin like that: one bouncer declared (in stream mode) and used by multiple routers. Do you agree this usage? This configuration is useful in "stream" mode (and only in this mode).

# static config
[experimental.plugins.bouncer]
  moduleName = "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
  version = "v1.1.13"
# Declaration of the unique bouncer
  [http.middlewares.crowdsec-traefik.plugin.bouncer]
    enabled = true                                      
    crowdseclapikey = "MY-LAPI-KEY"
    crowdsecMode = "stream"
    updateIntervalSeconds = 5
# Using this bouncer in a router (traefik file)
[http]
  [http.routers]
    [http.routers.app1]
      rule = "Host(`host.domain.fr`)"
      service = "app1"
      middlewares = ["crowdsec-traefik@file"]
      ....
# Using in docker containers label
  ......  
  labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app2.rule=Host(`my-app2.domain.fr`)"
      - "traefik.http.routers.app2.middlewares=crowdsec-traefik@file"
       ......
mathieuHa commented 1 year ago

Thank you for the reply. I deployed traefik and crowdsec-plugin like that: one bouncer declared (in stream mode) and used by multiple routers. Do you agree this usage? This configuration is useful in "stream" mode (and only in this mode).

# static config
[experimental.plugins.bouncer]
  moduleName = "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
  version = "v1.1.13"
# Declaration of the unique bouncer
  [http.middlewares.crowdsec-traefik.plugin.bouncer]
    enabled = true                                      
    crowdseclapikey = "MY-LAPI-KEY"
    crowdsecMode = "stream"
    updateIntervalSeconds = 5
# Using this bouncer in a router (traefik file)
[http]
  [http.routers]
    [http.routers.app1]
      rule = "Host(`host.domain.fr`)"
      service = "app1"
      middlewares = ["crowdsec-traefik@file"]
      ....
# Using in docker containers label
  ......  
  labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app2.rule=Host(`my-app2.domain.fr`)"
      - "traefik.http.routers.app2.middlewares=crowdsec-traefik@file"
       ......

Hey @guillaumekoto

You configuration looks good!
You can check further that everything is working as expected:

  1. activate debug log on the plugin

    logLevel: "DEBUG"

    You can then see what IP are checked within the bouncer:

    DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/04 13:17:52 cache:SetDecision ip:45.116.117.156 isBanned:true duration:601164s
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/07 07:04:57 ServeHTTP ip:82.66.XXX.XX isTrusted:false
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/07 07:04:57 cache:GetDecision ip:82.66.XXX.XX
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/07 07:04:57 ServeHTTP:getDecision ip:82.66.XXX.XX isBanned:false cache:miss
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/07 07:05:21 cache:GetDecision ip:updated
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/07 07:05:21 cache:SetDecision ip:updated isBanned:false duration:59s
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/08/07 07:05:21 handleStreamCache:updated

  2. Use CSCLI in crowdsec to check that logs are parsed correctly

    Acquisition Metrics:
╭────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────╮
│               Source               │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │
├────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┤
│ file:/var/log/traefik/access.log   │ 3.16k      │ 3.16k        │ -              │ 390                    │
│ file:/var/log/traefik/access.log.1 │ 51.97k     │ 51.97k       │ 5              │ 7.90k                  │
╰────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────╯
    Local Api Metrics:
╭──────────────────────┬────────┬──────╮
│        Route         │ Method │ Hits │
├──────────────────────┼────────┼──────┤
│ /v1/alerts           │ POST   │ 208  │
│ /v1/decisions/stream │ GET    │ 6712 │
│ /v1/heartbeat        │ GET    │ 6712 │
│ /v1/watchers/login   │ POST   │ 115  │
╰──────────────────────┴────────┴──────╯

    Theses are from one running test instance, that has sent 208 alerts to the CAPI, and we see that traefik logs have been parsed correctly.

  3. Use the CSCLI to ban yourself temporarly and check if after a minute you are denied

docker exec crowdsec cscli decisions add --ip YOUR_PLUBLIC_IP -d 10m # this will be effective 10min
guillaumekoto commented 1 year ago

All is clear, thank you for this confirmation and the job for this great plugin! I love crowdsec and traefik. You builded the perfect bridge of this two world!