search

owasp-modsecurity / ModSecurity-nginx

ModSecurity v3 Nginx Connector
Apache License 2.0
1.49k stars 277 forks source link

Support for building with nginx configured with PCRE2 #260

Closed defanator closed 2 years ago

defanator commented 2 years ago

Related changes in upstream: https://github.com/nginx/nginx/commit/c6fec0b027569a4e0b1d8aaee7dea0f2e4d6052b https://github.com/nginx/nginx/commit/931acbf5bcd550af8613d131f4ba49e22e909efb

This is going to be a part of upcoming nginx/1.21.5 release scheduled for December 28, 2021.

defanator commented 2 years ago

Unfortunately, it doesn't work well when the connector module is built with PCRE2, nginx is built with PCRE2, and libmodsecurity is built with PCRE1:

start:      SUMMARY: AddressSanitizer: 59014 byte(s) leaked in 2698 allocation(s).
1st reload: SUMMARY: AddressSanitizer: 18741824 byte(s) leaked in 291932 allocation(s).
2nd reload: SUMMARY: AddressSanitizer: 37483648 byte(s) leaked in 583864 allocation(s).
3rd reload: SUMMARY: AddressSanitizer: 56225472 byte(s) leaked in 875796 allocation(s).
stop:       SUMMARY: AddressSanitizer: 74967296 byte(s) leaked in 1167728 allocation(s).

Full error log here: https://gist.github.com/defanator/de14eacd93eeb44a82c608d624702b85

Perhaps better option for now would be to continue building ModSecurity-nginx with PCRE1. It won't work in long-term however as PCRE1 is basically not supported anymore and eventually everything (including libmodsecurity) hopefully will migrate to PCRE2.

I'll leave this one open just in case if anyone would suggest other options.

defanator commented 2 years ago

Surprisingly, there are memory leaks when connector is built with PCRE1 and nginx is built with PCRE2:

start:      SUMMARY: AddressSanitizer: 59014 byte(s) leaked in 2698 allocation(s).
1st reload: SUMMARY: AddressSanitizer: 18014174 byte(s) leaked in 288592 allocation(s).
2nd reload: SUMMARY: AddressSanitizer: 36028348 byte(s) leaked in 577184 allocation(s).
3rd reload: SUMMARY: AddressSanitizer: 54042522 byte(s) leaked in 865776 allocation(s).
stop:       SUMMARY: AddressSanitizer: 72056696 byte(s) leaked in 1154368 allocation(s).

Full error log: https://gist.github.com/defanator/274356c4f0594331e9d128af898182ae

JFTR, here are the versions of all involved components:

ModSecurity-nginx: https://github.com/SpiderLabs/ModSecurity-nginx/commit/2497e6ac654d0b117b9534aa735b757c6b11c84f ModSecurity: https://github.com/SpiderLabs/ModSecurity/commit/52958fa73c04ad6bb924497fd3d767f6e5e59411 nginx: https://github.com/nginx/nginx/commit/1f01183b9e6658749934313fd72f7f16c1918b54

The environment was built from https://github.com/defanator/modsecurity-performance (Ubuntu 20.04 "focal", vagrant box generic/ubuntu2004, version 3.6.2).

UPDATE: finally, leaks are still there with module built with PCRE1 and nginx built with PCRE1, so something bad is definitely happening in connector + libmodsec combo. Also, the above numbers were gathered without any external load between nginx reloads. If e.g. nikto scanning tool is running in a cycle (while [ :: ]; do nikto -host localhost -root /modsec-full/ ; done), worker's memory consumption wents crazy with every next nginx reload, especially in case when connector is using PCRE2.