LibModSecurity kills nginx process after an Owasp CRS rule detect a mssql SQL Information Leakage

[BerKut81]

Hello friends, this is my first post over here, please have a little patience. I'm pretty new doing this stuff and I'm a little bit confused.

So, let's go straight to the point. I have a few questions: . Is LibModSecurity connector really killing nginx process after detect a mssql Information leak? I think there is nonsense on kill a web server after a MSSQL Information leakage is detected. I'm not sure if there is a bug at nginx, libmodsecurity or Owasp rules. I take a look to the rule itself and it is impossible to send a kill signal from there. I'm really not sure where ask for help. . Is possible that there is another player over here? (in addition of nginx, libmodsecurity or Owasp rules)

Let's add some info: . The rule has the id 951220 and is located at file RESPONSE-951-DATA-LEAKAGES-SQL.conf from OWASP_CRS/3.3.0 . After this nginx is killed, I have to restart manually the service. For now I figured out how to start automatically, but I don't want to stop and start my web server every time this happens. . I'm using nginx as a reverse proxy server (there are several nginx, apache and IIS behind this nginx reverse proxy).

nginx logs:

2021/08/28 20:44:01 [notice] 25051#25051: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:44:11 [notice] 25054#25054: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:44:11 [notice] 25054#25054: signal process started 2021/08/28 20:44:12 [notice] 24144#24144: signal 1 (SIGHUP) received from 25054, reconfiguring 2021/08/28 20:44:12 [notice] 24144#24144: reconfiguring 2021/08/28 20:44:12 [notice] 24144#24144: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:44:12 [notice] 24144#24144: using the "epoll" event method 2021/08/28 20:49:52 [notice] 24146#24146: signal 15 (SIGTERM) received from 1, exiting 2021/08/28 20:49:52 [notice] 24145#24145: signal 15 (SIGTERM) received from 1, exiting 2021/08/28 20:49:52 [notice] 24145#24145: exiting 2021/08/28 20:49:52 [notice] 24146#24146: exiting 2021/08/28 20:44:13 [notice] 25055#25055: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:49:52 [notice] 24146#24146: exit 2021/08/28 20:49:52 [notice] 24145#24145: exit 2021/08/28 20:49:55 [notice] 25108#25108: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:49:57 [notice] 25149#25149: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:51:22 [notice] 25173#25173: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:51:23 [notice] 25176#25176: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:51:25 [notice] 25179#25179: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:58:10 [notice] 25316#25316: ModSecurity-nginx v1.0.2 (rules loaded inline/local/remote: 0/10944/0) 2021/08/28 20:58:10 [notice] 25316#25316: using the "epoll" event method 2021/08/28 20:58:10 [notice] 25316#25316: nginx/1.21.1 2021/08/28 20:58:10 [notice] 25316#25316: built by gcc 9.3.0 (Ubuntu 9.3.0-10ubuntu2) 2021/08/28 20:58:10 [notice] 25316#25316: OS: Linux 5.8.0-1042-aws 2021/08/28 20:58:10 [notice] 25316#25316: getrlimit(RLIMIT_NOFILE): 1024:524288 2021/08/28 20:58:10 [notice] 25317#25317: start worker processes 2021/08/28 20:58:10 [notice] 25317#25317: start worker process 25318 2021/08/28 20:58:10 [notice] 25317#25317: start worker process 25319

The rule itself:

SecRule TX:sql_error_match "@eq 1" \ "id:951220,\ phase:4,\ block,\ capture,\ t:none,\ msg:'mssql SQL Information Leakage',\ logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\ tag:'application-multi',\ tag:'language-multi',\ tag:'platform-mssql',\ tag:'attack-disclosure',\ tag:'paranoia-level/1',\ tag:'OWASP_CRS',\ tag:'capec/1000/118/116/54',\ ctl:auditLogParts=+E,\ ver:'OWASP_CRS/3.3.0',\ severity:'CRITICAL',\ chain" SecRule RESPONSE_BODY "@rx (?i)(?:System.Data.OleDb.OleDbException|[Microsoft][ODBC SQL Server Driver]|[Macromedia][SQLServer JDBC Driver]|> "capture,\ setvar:'tx.outbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}',\ setvar:'tx.sql_injection_score=+%{tx.critical_anomaly_score}',\ setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

Any help is really appreciated.-

[martinhsv]

Hi @BerKut81 ,

No, it is never the intention of ModSecurity to kill the web server.

It's hard to tell exactly what is happening in your case, but here are some things you could consider:

• you could try increasing the nginx log level (see error_log in your nginx conf). It might give you a bit more information about what is happening
• you could try looking at the ModSecurity debug log for information (and increase that log level to '9' while reproducing the effect).

One thing that stands out with the rule in question, is that it is running a regular expression against the RESPONSE_BODY. This can be very resource intensive if the response body is large. This is just a guess really, but if your response body is very large, this rule could be triggering some resource threshold (memory? elapsed time?) that may be causing nginx itself or some other monitoring process to trigger the SIGTERM.

(By the way, I can see your version of the connector, but what version of libModSecurity are you using? 3.0.5?)

[BerKut81]

First of all, thanks for your help.

I will check all of your recommendations and let you all know.

By the way, I'm using LibModSecurity3 v3.0.4-1build1

[BerKut81]

Hello folks.

Nothing new until now, I'm concerned about this: This is just a guess really, but if your response body is very large, this rule could be triggering some resource threshold (memory? elapsed time?) that may be causing nginx itself or some other monitoring process to trigger the SIGTERM.

Is possible to increase a parameter to increase that threshold? These parameters are at nginx or libmodsecuirty? Can you recommend any of them (if exists)?

Thanks in advance.-

[martinhsv]

Hello @BerKut81 ,

Is possible to increase a parameter to increase that threshold? These parameters are at nginx or libmodsecuirty? Can you recommend any of them (if exists)?

I would suggest not leaping head. As I said, it was just a guess . It's generally a more fruitful exercise to at least narrow down a probable cause before trying to identify a concrete solution for that unknown cause.

Not only was it just a guess, it was a very non-specific guess. Even if it does turn out some limitation is being reached, we have no idea what that limitation is.

In other words, there are currently no 'that threshold' or 'These parameters' to consider because we simply don't know whether a) any limit is being exceeded at all and if so b) what that limit is.

[BerKut81]

Hello Martin, thanks for your help.

I just want to to say that after I changed nginx log level from notice to info and enabled libmodsecurity log and set to level 3, nginx just never shutdown again. There past a a week from I enabled logs and everything is worked as expected.

I think you can close the issue and if somethings happens, I will let you know.

Thank you very much.

[martinhsv]

I'm glad to hear that you're not having a problem any more.

I will close this as you suggest, but a couple of concluding remarks:

Firstly, I expect that it was probably not the increased logging that stopped the unexpected shutdowns from happening. I.e. I suspect that it's more of a coincidence than a cause of the change in behaviour.

However, suppose you decide to experiment with this in future (e.g. 10 trials of turning ModSecurity debug logging off and the problem returns, while turning ModSecurity debugging log off again always means the problem does not recur for the following week) ...

... if this happens reliably and consistently, then that would give extra weight to the possibility that some ModSecurity bug is involved. I would definitely be interested in hearing about that.

Secondly, if you choose to leave debug logging on (even at level 3), this will start to consume disk space at a possibly problematic rate -- depending on how busy your web site is. So, if you choose to leave ModSecurity debug logging on, you may want to keep an eye on that.