Integration with crowdsecurity/cs-nginx-bouncer

[Sparkxxx]

Is your feature request related to a problem? Please describe. I would like to add protection for sites to NPM and be able to block IP's based on reputation/scenario using Crowdsecurity https://github.com/crowdsecurity which is a log analyzer and reactive firewall, something like fail2ban but in a much modern architecture and suitable for the containers world and having a distributed banlist generated from users around the world.

The question/feature request is how/where to add the https://github.com/crowdsecurity/cs-nginx-bouncer module to nginx config. This bouncer leverages nginx lua's API, namely access_by_lua_file. New/unknown IPs are checked against crowdsec API, and if request should be blocked, a 403 is returned to the user, and put in cache. The exact desired action can be configured in Crowdsec, like displaying captcas etc.

Describe the solution you'd like I would like to have an easy way (example maybe) to deploy the bouncer without breaking things in npm.

Describe alternatives you've considered None so far since it might take you minutes to offer a solution and it would take me hours to tweak things and break them :)

Additional context Please have a look at https://crowdsec.net/ to understand what is crowdsec and how it works. I use jc21/nginx-proxy-manager:latest for NPM and https://github.com/crowdsecurity/crowdsec/tree/master/docker for Crowdsec containers analyzing the nginx logs allready written by NPM on disk (docker map volumes ./npm_data/logs/:/var/log/nginx/ and have a quick look at acquis.yaml which tells crowdsec which logs to parse, mine looks like this): `filenames:

• /var/log/nginx/*.log labels: type: nginx`

Thank you and hope you'd consider this as a valuable addition to NPM

[erdoukki]

https://github.com/NginxProxyManager/nginx-proxy-manager/pull/1875#issuecomment-1047956739

This is an automated message from CI:

Docker Image for build 5 is available on DockerHub as jc21/nginx-proxy-manager:github-pr-1875

Note: ensure you backup your NPM instance before testing this PR image! Especially if this PR contains database changes.

I am trying with this "PR"... ;-)

[erdoukki]

Working ! But the dockerhub image from the PR do not include the modifications for CrowdSec integration. Bad attempt. Badly the actual PR do not have any checks from CI which to take dockerhub for tests on arm64 ! @baudneo Why you're modifications proposal do not include arm64 ? I check quickly your code but I still don't find the reason.

[baudneo]

@erdoukki I have multi-arch built now. ModSec is installed and loaded by default. User needs to configure it to start detecting though. See the PR for config. https://github.com/NginxProxyManager/nginx-proxy-manager/pull/1867

docker pull baudneo/nginx-proxy-manager:latest

[erdoukki]

Thanks!

Just tested;

root@STARGATE:~# cscli bouncers add docker-crowdsec-npm-bouncer
Api key for 'docker-crowdsec-npm-bouncer':

   xxxx

Please keep this key since you will not be able to retrieve it!

/srv/NGINX/data/nginx-proxy-manager/crowdsec/crowdsec-openresty-bouncer.conf

...
API_KEY=xxxx
...

docker-compose down
docker-compose up -d

root@STARGATE:/srv/NGINX# docker-compose restart
Restarting npm-crowdsec ... done
root@STARGATE:/srv/NGINX# cscli bouncers list
---------------------------------------------------------------------------------------------------------------------------
 NAME                         IP ADDRESS  VALID  LAST API PULL         TYPE                       VERSION                  
---------------------------------------------------------------------------------------------------------------------------
 crowdsec-firewall-bouncer    ::1         ✔️      2022-03-02T09:42:38Z  crowdsec-firewall-bouncer  v0.0.21-openwrt-0.0.21-3 
 docker-crowdsec-npm-bouncer              ✔️      2022-03-02T09:53:29Z                                                      
---------------------------------------------------------------------------------------------------------------------------

but still issue :

time="02-03-2022 10:55:57" level=error msg="auth api key error: select bouncer: ent: bouncer not found: unable to query"

[erdoukki]

Okay, just rebuild the docker and now works fine:

INFO[02-03-2022 01:56:04 PM] Local Api Bouncers Metrics:                  
+-----------------------------+----------------------+--------+------+
|           BOUNCER           |        ROUTE         | METHOD | HITS |
+-----------------------------+----------------------+--------+------+
| crowdsec-firewall-bouncer   | /v1/decisions/stream | GET    | 1067 |
| docker-crowdsec-npm-bouncer | /v1/decisions/stream | GET    |   37 |
+-----------------------------+----------------------+--------+------+

[klausagnoletti]

I have added ModSecurity to a new image with CrowdSec

@baudneo Where can I submit issues for your fork? I have two: 1) Specifying enviromental variables like this doesn't work. The bouncer is not started unless I edit the crowdsec-openresty-bouncer.conf file.

    environment:
      CROWDSEC_BOUNCER: "1"
      CROWDSEC_LAPI: "http://1xxxx:8080"
      CROWDSEC_KEY: "xxxx"

2) There's a memory leak in the current openresty bouncer that currently is fixed with a workaround. Will you implement it in your fork?

Lastly I have a suggestion: Could you implement more environmental variables so everything in the bouncer configuration can configured like this. Especially the reCAPTCHA stuff..

Thanks - keep up the good work!

[h4llm3n]

I have added ModSecurity to a new image with CrowdSec

Now CrowdSec and ModSecurity are installed and working. The new image that has modsec is docker pull baudneo/nginx-proxy-manager:cs-modsec. I installed OWASP-CRS for modsec. The modsec audit log is in /data/logs/modsec_audit.log by default, you can turn it off by editing /data/modsec/modsecurity.conf.

NOTE

• All config for modsec is in /data/modsec which is symbolically linked to /etc/nginx/modsec
• The OWASP rules are in /data/modsec/ruleset/
• The main rules file is /data/modsec/main.conf which just Includes the OWASP ruleset.
• The only ENV VAR for modsec is MODSEC_CREATE which will force create the directory structure into /data/modsec. This is usually only run on the first run of the modsec image when this directory does not exist. this ENV VAR will allow you to force it for whatever reason.

To enable modsec you need to add these 2 directives to the config somewhere.

modsecurity on;
modsecurity_rules_file /etc/nginx/modsec/main.conf;

Here are some pointers.

• To enable modsec for ALL HTTP hosts put the above directives in /data/nginx/custom/http_top.conf.
• To enable modsec for ALL STREAM hosts put the above directives in /data/nginx/custom/stream.conf.
• To enable modsec for ONE certain HTTP host put the above directives in the 'Advanced' tab configuration.
• To enable modsec for ONE certain HTTP host in a certain LOCATION put the above directives in the location {} block instead of outside of it in the 'Advanced' tab config.
• To enable on a per-stream basis you need to enable the streams in the dashboard and then edit the stream file by hand to add those directives into the stream {} block you want.

Some logs from modsec_audit.log to confirm modsecurity is working. This was a test using ?exec=/bin/bash in the URI.

---C3KCgGTx---F--
HTTP/2.0 403
Server: nginx
Date: Tue, 22 Feb 2022 06:16:02 GMT
Content-Length: 552
Content-Type: text/html
Connection: close
Strict-Transport-Security: max-age=63072000;includeSubDomains; preload

---C3KCgGTx---H--
ModSecurity: Warning. Matched "Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:exec' (Value: `/bin/bash' ) [file "/etc/nginx/modsec/ruleset/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "480"] [id "932160"] [rev ""] [msg "Remote Command Execution: Unix Shell Code Found"] [data "Matched Data: bin/bash found within ARGS:exec: /bin/bash"] [severity "2"] [ver "OWASP_CRS/3.3.2"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "172.18.0.4"] [uri "/zm/"] [unique_id "1645510562"] [ref "o1,8v14,9t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/etc/nginx/modsec/ruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "80"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.3.2"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "172.18.0.4"] [uri "/zm/"] [unique_id "1645510562"] [ref ""]

I don't know JavaScript so I am unable to add to the UI to make things pretty. Ideally there would be a ModSecurity tab that has a toggle switch for on/off and a text input box for the rules file. This tab would be in the proxyhost modal where you set what domain/target host/target port etc. It would be nice to have some sort of UI for corwdsec as well to turn it off and on and to edit the config file. I don't think having a rule editor for modsec would matter much, but it would be nice to have it.

I might take the time to teach myself some JS just to implement this to make it all pretty. Things are working for CrowdSec and ModSecurity though. The people who want that functionality should not have an issue with configuring things from the CLI anyways, for now.

Do I still have to add the log path to /etc/crowdsec/acquis.yaml here?

[klausagnoletti]

Awesome! Thanks for updating it!

[luismanson]

Hello baudneo, thanks for this new image. I've tried to update, however docker can't pull the image, it complains about missing manifest. I've verified docker hub and think the image is not updated, is that ok?

$ docker pull baudneo/nginx-proxy-manager:cs-modsec Error response from daemon: manifest for baudneo/nginx-proxy-manager:cs-modsec not found: manifest unknown: manifest unknown

[shanelord01]

Any chance this is being merged back into the base jc21/nginx-proxy-manager image ?

[baudneo]

ModSec is EoL now/soon and also has known memory leaks which are HUGE, so I won't be building images with it. I am currently working on ZoneMinder ML object detection stuff, as soon as I am done that I will create a new image with current NPM and add crowdsec openresty bouncer into it.

I think the options for WAF are slim, coraza and open-appsec which I may or may not look into adding either of them into an NPM image.

Coraza parses modsec security language, so the rules files like OWASP (who develop coraza as well) lists can be used with it. Furthermore, Coraza seems to need to be implemented in a Go language app. It is not as simple as download, point it at configs/rules and run it, someone needs to write a Go app using coraza libraries. That may have changed as I have not kept up with Coraza.

open-appsec uses ML instead of signatures for threat detection which is neat but, I can't speak to its effectiveness. At the time when I realized ModSec had issues, open-appsec did not have a nginx connector, which I think they do now. They require you to have a cloud control panel of sorts, so IDK how feasible it is to integrate into NPM if you can't do a stand alone install. Again, things may have changed as I haven't kept up on open-appsec either.

I stopped using NPM in favour of Cloudflare tunnels but, CF's new browser security check stuff is causing me grief so, I will need to move back to NPM.

[SeeJayEmm]

Is there any hope of this being revived? I would love to have the option of integrating Crowdsec with NPM.

[domodial]

@baudneo

I have the following problem. When I want to login to npm and change the email as it asks me, the response is Forbidden.

If I want to create a new user, I can't apply rights or change the password = Forbidden. I saw this problem in a very old version of npm without having been confronted with the problem. But I read it on the web.

I have a working npm instance which I temporarily shut down, Crowdesc installed with no errors. The changes made in the conf files are ok, I created a separate base for this fork baudneo/nginx-proxy-manager:latest in mariadb.

My concern is only on the change of email and password. So I tell myself that further on it might not be so good as well.

Note : The correct permissions are applied to the folders in your fork. (I'm under unraid on Dell PowerEdge)

[github-actions[bot]]

Issue is now considered stale. If you want to keep it open, please comment :+1:

[caseyjhol]

👍