search

owasp-modsecurity / ModSecurity

ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx. It has a robust event-based programming language which provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis.
https://www.modsecurity.org
Apache License 2.0
7.67k stars 1.54k forks source link

How to disable some logs? #3110

Closed rahulthackkar closed 1 month ago

rahulthackkar commented 1 month ago

Hello Support,

I have installed mod security 2 in apache (ubuntu 22.04).

I just wanted to limit log file size by removing

I tried to find out configuration for this, but could not.

My Log parts are only 'H'

SecAuditLogParts H

Sample logs are below I want to avoid.

--e5e05673-A-- [22/Mar/2024:06:15:41.533563 +0000] Zf0iDb5-6jzuz1NaSnMeLwABUAw 93.86.97.137 60415 172.31.3.102 443 --e5e05673-H-- Apache-Handler: proxy:fcgi://maskeddomainname.com Stopwatch: 1711088141472444 61130 (- - -) Stopwatch2: 1711088141472444 61130; combined=651, p1=640, p2=0, p3=0, p4=0, p5=11, sr=119, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache

--e5e05673-Z--

--62aff97c-A-- [22/Mar/2024:06:15:58.021859 +0000] Zf0iHqWEDu3IDZ7ZKiTwvAAAAQo 114.119.145.163 49191 172.31.0.20 443 --62aff97c-E--

--62aff97c-H-- Apache-Error: [file "mod_access_compat.c"] [line 350] [level 3] AH01797: client denied by server configuration: /home/maskeduser/public_html/a6-838 Stopwatch: 1711088158019766 2111 (- - -) Stopwatch2: 1711088158019766 2111; combined=1298, p1=812, p2=0, p3=83, p4=197, p5=205, sr=136, sw=1, l=0, gc=0 Response-Body-Transformed: Dechunked Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache Engine-Mode: "DETECTION_ONLY"

--62aff97c-Z--

--8b7f4f31-A-- [22/Mar/2024:11:49:34.742782 +0000] Zf1wTgL-oH_rxEQ3Oeq6JwAAAEc 185.234.216.114 56180 172.31.15.118 80 --8b7f4f31-E--

--8b7f4f31-H-- Apache-Handler: proxy:fcgi://maskeddomain.com Stopwatch: 1711108174739077 3739 (- - -) Stopwatch2: 1711108174739077 3739; combined=1566, p1=560, p2=681, p3=56, p4=129, p5=140, sr=104, sw=0, l=0, gc=0 Response-Body-Transformed: Dechunked Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache Engine-Mode: "DETECTION_ONLY"

--8b7f4f31-Z--

Can you please help me with this?

airween commented 1 month ago

Hi @rahulthackkar,

thanks for reporting this.

Please take a look to Reference: parts A and Z are mandatory.

The existence of part E header is perhaps a bug, we have to investigate it.

rahulthackkar commented 1 month ago

Hi @airween,

Thanks for quick response, I know A and Z are mandatory, even after specifying only 'H' It is logging A and Z, I have no issue with this.

I want to disable whole log entry for above listed types which is no meaningful to me.

How can I do that?

dune73 commented 1 month ago

What do you need the audit log for? In most use cases the error-log brings all you need to have and if you reduce the audit log to part H, then there is no added value. I would simply disable the audit log.

rahulthackkar commented 1 month ago

What do you need the audit log for? In most use cases the error-log brings all you need to have and if you reduce the audit log to part H, then there is no added value. I would simply disable the audit log.

Let me try to explain my concern to you .

I have just enabled mod_security on my server which have 100s of websites. As I can not enable it right away due to strict rule set, I have kept engine on detection mode only.

So what I am trying to figure out is, what my application really needs anyway, and what are actual vulnerabilities I should restrict to enter.

For that reason , daily I need to observe logs for it, and One by one I am writing exclusions according to my application need.

Keeping only H part serves both the concerns, keeping disk usage low, and I got triggered rule info too.

Eventually I will turn on security engine once I covered all aspects, so for that I observed that , some of logs I don't need actually.

Is there any way to disable such log entries?

I was referring this : https://github.com/owasp-modsecurity/ModSecurity/issues/329

dune73 commented 1 month ago

This is at best a compile time flag and no config time flag. Also I have never seen it used.

But again: Why do you run the audit log at all? If you are only interested in the alerts in the H part, then you have everything in the error log. In other words :

Auditlog-Part-H-Alert-Messages == Errorlog-Alert-Messages

(Well, plus a timestamp, but that's OK I guess.)

rahulthackkar commented 1 month ago

Hello @dune73

can you explain me what this below log is for?

--e5e05673-A-- [22/Mar/2024:06:15:41.533563 +0000] Zf0iDb5-6jzuz1NaSnMeLwABUAw 93.86.97.137 60415 172.31.3.102 443 --e5e05673-H-- Apache-Handler: proxy:fcgi://maskeddomainname.com Stopwatch: 1711088141472444 61130 (- - -) Stopwatch2: 1711088141472444 61130; combined=651, p1=640, p2=0, p3=0, p4=0, p5=11, sr=119, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache

--e5e05673-Z--

I am not able to understand it even.

Also response body dechunked logs I want to avoid, so I can actually observe other logs those are actually triggered by vulnerability critical or warning

dune73 commented 1 month ago

If you do not understand the Stopwatch stuff, you should turn to the documentation (if you really want to know). As for the rest, it's pretty self-explanatory.

rahulthackkar commented 1 month ago

I just want to disable some logs , is there any option?

rahulthackkar commented 1 month ago

For example, client denied by server configuration, this error log is being logged in both files mod sec audit log and apache error log, before mod security it was already being logged into apache error log.

dune73 commented 1 month ago

You can disable the SecAuditEngine. In fact this is what I have been trying to explain to you for several days now.

rahulthackkar commented 1 month ago

By disabling SecAuditEngine , it will disable whole audit log?

If yes then that is not I am looking for.

On Mon, 25 Mar, 2024, 12:52 pm Christian Folini, @.***> wrote:

You can disable the SecAuditEngine. In fact this is what I have been trying to explain to you for several days now.

— Reply to this email directly, view it on GitHub https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2017372699, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEALHM442FYUPMKHRB45ZJLYZ7GEFAVCNFSM6AAAAABFDF3L6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJXGM3TENRZHE . You are receiving this because you were mentioned.Message ID: @.***>

airween commented 1 month ago

By disabling SecAuditEngine , it will disable whole audit log? If yes then that is not I am looking for.

Here you wrote:

I just want to disable some logs , is there any option?

There are three kind of logs:

So, what other log do you want to disable?

rahulthackkar commented 1 month ago

in audit log, I want to disable some logs like response body dechunked, apache error logs which are already being logged in apache's error log, also logging in mod sec audit log, stopwatch logs.

On Mon, 25 Mar, 2024, 7:42 pm Ervin Hegedus, @.***> wrote:

By disabling SecAuditEngine , it will disable whole audit log? If yes then that is not I am looking for.

Here https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2017347952 you wrote:

I just want to disable some logs , is there any option?

There are three kind of logs:

  • error.log - produced by HTTP server, you can turn of in server config, or can modify the severity; please read the server documentation
  • audit.log - see @dune73 https://github.com/dune73's comments
  • debug.log - that's disabled by default

So, what other log do you want to disable?

— Reply to this email directly, view it on GitHub https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2018096931, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEALHM3CHK5HZ7TEDIYBB7LY2AWDTAVCNFSM6AAAAABFDF3L6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYGA4TMOJTGE . You are receiving this because you were mentioned.Message ID: @.***>

dune73 commented 1 month ago

I'm out.

rahulthackkar commented 1 month ago

Thanks @Christian for your efforts, I think i am not able to convey it properly what I am looking for.

On Mon, 25 Mar, 2024, 8:18 pm Christian Folini, @.***> wrote:

I'm out.

— Reply to this email directly, view it on GitHub https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2018176393, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEALHM5XGQISLQCCB5YSFQLY2A2LRAVCNFSM6AAAAABFDF3L6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYGE3TMMZZGM . You are receiving this because you were mentioned.Message ID: @.***>

rahulthackkar commented 1 month ago

Can anyone help me to disable below audit log only, not whole audit log?

--8b7f4f31-A-- [22/Mar/2024:11:49:34.742782 +0000] Zf1wTgL-oH_rxEQ3Oeq6JwAAAEc 185.234.216.114 56180 172.31.15.118 80 --8b7f4f31-E--

--8b7f4f31-H-- Apache-Handler: proxy:fcgi://maskeddomain.com Stopwatch: 1711108174739077 3739 (- - -) Stopwatch2: 1711108174739077 3739; combined=1566, p1=560, p2=681, p3=56, p4=129, p5=140, sr=104, sw=0, l=0, gc=0 Response-Body-Transformed: Dechunked Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache Engine-Mode: "DETECTION_ONLY"

--8b7f4f31-Z--

dune73 commented 1 month ago

Maybe you should post an example of the desired audit log.

Also: "log" or "logs" always means the entire log file or log type. You should use the term "item" or "record" for items within the audit log or withing an audit log part.

rahulthackkar commented 1 month ago

I want to avoid whole entry by its type, not part(s) of one audit log.

On Mon, 25 Mar, 2024, 8:40 pm Christian Folini, @.***> wrote:

Maybe you should post an example of the desired audit log.

Also: "log" or "logs" always means the entire log file or log type. You should use the term "item" or "record" for items within the audit log or withing an audit log part.

— Reply to this email directly, view it on GitHub https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2018230495, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEALHM3FVJOZFVIXWNRX4QDY2A453AVCNFSM6AAAAABFDF3L6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYGIZTANBZGU . You are receiving this because you were mentioned.Message ID: @.***>

airween commented 1 month ago

I want to avoid whole entry by its type, not part(s) of one audit log.

A bit more concrete...?

As @dune73 wrote: "Maybe you should post an example of the desired audit log."

I still don't see the expected pattern.

rahulthackkar commented 1 month ago

Sure let me try again.

Below is one audit log entry in mod sec audit log I want to avoid, may be about response body transform dechunk event.

--8b7f4f31-A-- [22/Mar/2024:11:49:34.742782 +0000] Zf1wTgL-oH_rxEQ3Oeq6JwAAAEc 185.234.216.114 56180 172.31.15.118 80 --8b7f4f31-E--

--8b7f4f31-H-- Apache-Handler: proxy:fcgi://maskeddomain.com Stopwatch: 1711108174739077 3739 (- - -) Stopwatch2: 1711108174739077 3739; combined=1566, p1=560, p2=681, p3=56, p4=129, p5=140, sr=104, sw=0, l=0, gc=0 Response-Body-Transformed: Dechunked Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache Engine-Mode: "DETECTION_ONLY"

--8b7f4f31-Z--

another log entry which typically involves only stopwatch is below

--e5e05673-A-- [22/Mar/2024:06:15:41.533563 +0000] Zf0iDb5-6jzuz1NaSnMeLwABUAw 93.86.97.137 60415 172.31.3.102 443 --e5e05673-H-- Apache-Handler: proxy:fcgi://maskeddomainname.com Stopwatch: 1711088141472444 61130 (- - -) Stopwatch2: 1711088141472444 61130; combined=651, p1=640, p2=0, p3=0, p4=0, p5=11, sr=119, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/); OWASP_CRS/3.3.2. Server: Apache

--e5e05673-Z--

I also don't want above log entry in audit log.

Is there any option in mod security to avoid specific log entries??

rahulthackkar commented 1 month ago

May be only those log entries which are actually triggered by rules only(like sql injection, xss attack, code injection) etc.. Hope this explains my concern enough.

airween commented 1 month ago

Is there any option in mod security to avoid specific log entries??

you mean remove some headers from an audit log part?

No, I don't think if it is possible.

And let me ask you something - in your official post, you asked:

I just wanted to limit log file size by removing

You really want to reduce your log with remove specific lines...?

rahulthackkar commented 1 month ago

Not headers only, above two log entries I shared with you previously, I want to avoid such whole log entries.

On Mon, 25 Mar, 2024, 10:47 pm Ervin Hegedus, @.***> wrote:

Is there any option in mod security to avoid specific log entries??

you mean remove some headers from an audit log part?

No, I don't think if it is possible.

And let me ask you something - in your official post, you asked:

I just wanted to limit log file size by removing

You really want to reduce your log with remove specific lines...?

— Reply to this email directly, view it on GitHub https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2018509481, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEALHM2KXKGC3NEGQ2QR6BLY2BL3LAVCNFSM6AAAAABFDF3L6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYGUYDSNBYGE . You are receiving this because you were mentioned.Message ID: @.***>

airween commented 1 month ago

You can control conditionally the logparts runtime with ctl:auditLogParts ctl action, eg:

SecRule YOUR_CONDITION \
    "id:NNNNN,\
    phase:N,\
    t:none,\
    nolog,\
    pass,\
    ctl:auditLogParts=-E"

See the examples on reference page, eg this one.

rahulthackkar commented 1 month ago

I want to disable completely above log entries, I am not sure how I can write condition for those? as they are not any rule triggered, they are just showing that response body transform dechunked.

How can I do this?

On Tue, 26 Mar, 2024, 12:34 am Ervin Hegedus, @.***> wrote:

You can control conditionally the logparts runtime with ctl:auditLogParts ctl action, eg:

SecRule YOUR_CONDITION \ "id:NNNNN,\ phase:N,\ t:none,\ nolog,\ pass,\ ctl:auditLogParts=-E"

See the examples on reference page, eg this one https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-(v2.x)#accuracy .

— Reply to this email directly, view it on GitHub https://github.com/owasp-modsecurity/ModSecurity/issues/3110#issuecomment-2018706447, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEALHMY7VD2KRBLTHB2VX23Y2BYLTAVCNFSM6AAAAABFDF3L6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYG4YDMNBUG4 . You are receiving this because you were mentioned.Message ID: @.***>

airween commented 1 month ago

I want to disable completely above log entries, I am not sure how I can write condition for those?

you wrote above:

May be only those log entries which are actually triggered by rules only(like sql injection, xss attack, code injection)

so you can copy the rule's condition or you can append the ctl:auditLogParts=-E or what you want to the rule.

as they are not any rule triggered, they are just showing that response body transform dechunked. How can I do this?

As we explained above, YOU CAN'T. There is no way to remove lines from an audit log part. You can control only parts (except mandatory ones).

rahulthackkar commented 1 month ago

Okay, Thanks. I will work around my way.