lua-resty-waf - lua based WAF - performance similar to cloudflare WAF

[adrelanos]

Are you aware of lua-resty-waf by @p0pr0ck5? A LUA based WAF.

Performance

lua-resty-waf was designed with efficiency and scalability in mind. It leverages Nginx's asynchronous processing model and an efficient design to process each transaction as quickly as possible. Load testing has show that deployments implementing all provided rulesets, which are designed to mimic the logic behind the ModSecurity CRS, process transactions in roughly 300-500 microseconds per request; this equals the performance advertised by Cloudflare's WAF. Tests were run on a reasonable hardware stack (E3-1230 CPU, 32 GB RAM, 2 x 840 EVO in RAID 0), maxing at roughly 15,000 requests per second. See this blog post for more information.

lua-resty-waf workload is almost exclusively CPU bound. Memory footprint in the Lua VM (excluding persistent storage backed by lua-shared-dict) is roughly 2MB.

Cloudflare WAF is LUA based.

https://blog.cloudflare.com/cloudflares-new-waf-compiling-to-lua/

Unfortunately lua-resty-waf hasn't seen development (git commits) since 2018.

Would it make sense to rebase ModSecurity-nginx on lua-resty-waf?

Consider hire p0pr0ck5 if he's not hired by some CDN / WAF company yet? :)

I am writing this on my own initiative. Didn't talk to p0pr0ck5.

[zimmerle]

ModSecurity supports a language name SecRules language, atop of that, it also natively supports Lua. ModSecurity is used with Apache, Ngnix, LiteSpeed, and other commercial applications. There are also binding for Python and other scripts languages (via swig). We are aiming to keep being the world's most deployed and resilient WAF. What do you consider to be the aspects that we should adopt from lua-resty-waf?

[adrelanos]

Thank you for your reply! As far as I understand:

• SpiderLabs/ModSecurity-nginx is implemented in C.
• lua-resty-waf is implemented in LUA.

It's a bold suggestion to make, "consider dropping your C code base and re-implement in LUA (perhaps fork lua-resty-waf)".

ModSecurity is used with Apache, Ngnix, LiteSpeed, and other commercial applications.

Which makes the success for my suggestion even less likely - because LUA based implementation lua-resty-waf is Nginx-only. Unless you'd have capacity to maintain two different implementations at the same time.

What do you consider to be the aspects that we should adopt from lua-resty-waf?

As far as I understand lua-resty-waf is said to have better performance. Even comparable with the biggest commercial CDN (cloudflare) currently.

I guess please interpret my post as "Have you heard of lua-resty-waf yet?". "Anything interesting there? Does it provide inspiration which could help here?"

[zimmerle]

We like the idea to provide a WAF that is multi propose (or Swiss Army knives, quoting Ryan Barnett). We understand that this is a bold goal. Still, we try to keep the engine with excellent performance and yet, with the flexibility to perform a complex inspection (less performant and consequently more computationally intensive) if necessary.

One may misinterpret a set of very computationally intensive rules with ModSecurity being slow. The concrete fact is that ModSecurity is as fast as the rules allow it to be. Likewise, perform computational intensive stuff in Lua won't help a Lua WAF.

Of course, the engine is suitable to compute things with different approaches, more or less performant or even more or less performant in a given particular use case scenario. Said that there is a lot of space for performance improvement within ModSecurity and as an Open Source project, we incentive people co contribute and participate in the development. That is why I think you made a fascinating question.

For the ones who already have the @agentzh OpenResty running, it will be an excellent idea to use a Lua WAF. That Lua WAF could very well be a Lua connector for libModSecurity or even accessing the evasion functions straight from libModSecurity. This composition will make a very capable WAF - in terms of features - and yet more performant in the sense that it will be integrated into an infra that already exists.

I hope I have addressed your concern.

[jeremyjpj0916]

Robert is a busy dude, he works for Kong, great guy very smart. Doesn't get any free time for WAF fun these days it seems. I would be curious what the performance of lua-resty-waf running the CRS is VS ModSecurity v3 running CRS and the comparable performance.

I've noticed one of the slower issues with ModSecurity and CRS is the way it parses and checks malicious snippits in the HTTP Body. since the engine breaks down xml/json into a big map of key values and then runs regex against them all separately big json/xml blobs end up taking time to have all key value pairs have iterative regex parsing done against them. Idk what the fastest algo's out there are to evaluate json/xml for real attack threat vectors but I imagine there must be faster ways certain commercial waf's approach payload inspection(do they even parse it or just evaluate the body as one big string read in memory? Would it be faster? Idk.

[adrelanos]

but I imagine there must be faster ways certain commercial waf's approach payload inspection(do they even parse it or just evaluate the body as one big string read in memory?

In this youtube video Building a low-latency WAF inside NGINX using Lua: John Graham-Cumming @nginxconf 2014 a cloudflare engineer describes how their (I think proprietary, closed source, SaaS) LUA based WAF optimizes performance. This might answer your question and also be generally interesting.

[zimmerle]

I've noticed one of the slower issues with ModSecurity and CRS is the way it parses and checks malicious snippits in the HTTP Body. since the engine breaks down xml/json into a big map of key values and then runs regex against them all separately big json/xml blobs end up taking time to have all key value pairs have iterative regex parsing done against them. Idk what the fastest algo's out there are to evaluate json/xml for real attack threat vectors but I imagine there must be faster ways certain commercial waf's approach payload inspection(do they even parse it or just evaluate the body as one big string read in memory? Would it be faster? Idk.

@jeremyjpj0916 without knowing your use case scenario is hard to address an eventual issue or clarify a configuration problem. Give us better details. Preferable in a different issue. ModSecurity is flexible enough to treat the payload with a different approach depending on its characteristics; up for the rules.

[zimmerle]

In this youtube video Building a low-latency WAF inside NGINX using Lua: John Graham-Cumming @nginxconf 2014 a cloudflare engineer describes how their (I think proprietary, closed source, SaaS) LUA based WAF optimizes performance. This might answer your question and also be generally interesting.

@adrelanos, one year later (2015) we have presented at nginx.conf "ModSecurity and NGINX - The Ideal Security Environment For Your Web Application" the talk you are referring to was considered.

[jeremyjpj0916]

without knowing your use case scenario is hard to address an eventual issue or clarify a configuration problem. Give us better details. Preferable in a different issue. ModSecurity is flexible enough to treat the payload with a different approach depending on its characteristics; up for the rules.

My use case is generic APIs (NGINX as a reverse proxy gateway), REST/SOAP/Websocket(the handshake anyways) etc. HTTP Header and HTTP Body evaluation of all shapes and sizes. Its indeed true ModSecurity does not write the rules, any maybe they are the reason for general slowness. Probably fits the bill of most ModSecurity use cases, the passage of HTTP data back and forth. Another thing I have noticed is high variance on latency with ModSecurity v3 + CRS enabled. If my company gives me a few months I will try to clean up the lua-rest-waf as a Kong plugin and see if I can convert some of the CRS ruleset into it and run those requests against some API data (json/xml and so forth) and see how fast each are. No promises they will let me play around with it though as we just are wrapping up ModSecurity as our waf solution(more widely adapted, already production ready it seems besides some slowness here and there we see in performance(again might not always be "modsecurity" but the rules that run within it and the way the rules have to adhere to the engines capabilities).

@adrelanos I found the slides from that video too: https://github.com/cloudflare/jgc-talks/blob/master/nginx.conf/2014/cloudflare-lua-waf.pdf

One thing I will say is I have trouble believing folks run 6000+ rules on a payload(say 10kb+ headers and bodies concurrently) and expect 1ms latency, I think these kinda numbers are false(maybe if I tested it myself and saw it I would be a converted believer heh). Like maybe they ran it on a 500 byte request or something very small lol. At least with ModSec v3 running CRS I see a 10-50+ms hit on 10kb+ payloads with high variance. Not CPU or Mem capped either, plenty of available resources on node.

[adrelanos]

Felipe Zimmerle:

@adrelanos, one year later (2015) we have presented at nginx.conf "ModSecurity and NGINX - The Ideal Security Environment For Your Web Application" the talk you are referring to was considered.

Is there any recording of this on youtube or somewhere? I'd like to watch.

[zimmerle]

Is there any recording of this on youtube or somewhere? I'd like to watch it.

Not that I know by heart. There is, however, the blog post, which addresses some of the subjects that were presented during the talk.

Over the years, we had evolved the technical details presented in the Nginx talk; those new technical details had been filed in assorted security conferences. If you are concerned about performance, for instance, the last BlackHat, @victorhora, and I have presented (among other things), Modsecurity running in a badge.

[zimmerle]

@adrelanos @jeremyjpj0916 did you guys had the chance actually to proceed with the performance tests?

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days