search

owasp-modsecurity / ModSecurity

ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx. It has a robust event-based programming language which provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis.
https://www.modsecurity.org
Apache License 2.0
7.7k stars 1.54k forks source link

Final boundary missing with Docushare #172

Closed rcbarnett-zz closed 10 years ago

rcbarnett-zz commented 10 years ago

MODSEC-19: Docushare (from Xerox) has a "native" win32 client that allows users to "mount" Docushare servers - so they can drag-n-drop. I think it was sort of a precursor to WebDAV? From what I've seen, it reminds me a bit of a Windows subversion client app - TortoiseSVN

Anyway, I am seeing modsec blocking users from using this native app. I see "ModSecurity: Multipart parsing error: Multipart: Final boundary missing" trigger, followed directly by 960912 - "Warning. Match of "eq 0" against "REQBODY_PROCESSOR_ERROR" required"

I'll attach the modaudit dump of the event - after removing the content.

[Sun Sep 14 09:08:57 2008] [error] [client 192.168.0.182] ModSecurity: Warning. Match of "eq 0" against "REQBODY_PROCESSOR_ERROR" required. [file "/etc/httpd/modsecurity.d/modsecurity_crs_20_protocol_violations.conf"] [line "31"] [id "960912"] [msg "Request Body Parsing Failed. Multipart parsing error: Multipart: Final boundary missing."] [severity "CRITICAL"] [hostname "hst.domain.com"] [uri "/docushare/dscgi/ds.py/ApplyUpload/File-287346"] [unique_id "dsx7wQoBPEEAADLnULsAAAAy"] [Sun Sep 14 09:09:27 2008] [error] [client 192.168.0.182] ModSecurity: Multipart parsing error: Multipart: Final boundary missing. [hostname "hst.domain.com"] [uri "/docushare/dscgi/ds.py/ApplyUpload/File-287347"] [unique_id "eDBmKQoBPEEAADLnULwAAAAy"]

rcbarnett-zz commented 10 years ago

Original reporter: jasonhaar

rcbarnett-zz commented 10 years ago

jasonhaar: You are right - there isn't an empty line at the end of the boudary - the Docushare client is broken.

Well - it isn't - as it is only defined as working with it's own server - so it really doesn't have to abide by HTTP standards at all :-/

If you can figure out a way that I can whitelist this, it would be appreciated.

Thanks

Jason

rcbarnett-zz commented 10 years ago

brectanus: Perhaps we can just look at the end of the buffer when we hit the end of the stream (ie just before we flag a missing final boundary) and see if the final boundary is there (sans whitespace)? If it is, then mark it there and throw a different flag marking a malformed final boundary?

rcbarnett-zz commented 10 years ago

nicksoulliere: I have a similar issue, seeing "ModSecurity: Multipart parsing error: Multipart: Final boundary missing" for a proprietary piece of software we use. Instead of trying to get the developers to fix it, having a way to create an exception for this particular case would be very beneficial.

rcbarnett-zz commented 10 years ago

brectanus: Nick,

Please attach an audit log entry to this issue so we can take a look and see what the parser may be missing. It may not be the same problem as with Jason.

rcbarnett-zz commented 10 years ago

nicksoulliere: Of course, sorry for not providing it earlier:

[12/Sep/2008:16:09:34 --0400] [xxx.eprize.net/sid#2ba6d2a29d50][rid#2ba6d405ae38][/xxx/yyy/contest_image_upload_xxx_yyy][2] Multipart parsing error: Multipart: Final boundary missing. [12/Sep/2008:16:09:34 --0400] [xxx..eprize.net/sid#2ba6d2a29d50][rid#2ba6d405ae38][/xxx/yyy/contest_image_upload_xxx_yyy][1] Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_PROCESSOR_ERROR" required. [file "/etc/httpd/modsecurity.d/modsecurity_crs_20_protocol_violations.conf"] [line "31"] [id "960912"] [msg "Request Body Parsing Failed. Multipart parsing error: Multipart: Final boundary missing."] [severity "CRITICAL"]

rcbarnett-zz commented 10 years ago

nicksoulliere: I will attach a raw data dump from the mod_security console as well in a few minutes.

rcbarnett-zz commented 10 years ago

brectanus: Nick,

Yours is a bit different. This POST request really is missing the final boundary. But, it does not appear to be because of a malformed request, but rather a truncated request. The Content-Length header states the body should have been 724869 bytes, but the request body in the audit log entry is only 29040 bytes long (missing 695829 bytes). So, it looks like maybe the upload was just canceled?

rcbarnett-zz commented 10 years ago

nicksoulliere: Thanks Brian, I am sure you are correct. While I work this out with the developers, is there anything I can do on my end to stop logging these particular requests?

rcbarnett-zz commented 10 years ago

ivanr: You could try to write a rule that would check REQBODY_PROCESSOR_ERROR_MSG for this particular problem and turn off the audit log engine. There is no way to not see the message in the error log, though.

I've just proposed MODSEC-21 to deal with the problem.

rcbarnett-zz commented 10 years ago

ivanr: Brian wrote:

Perhaps we can just look at the end of the buffer when we hit the end of the stream (ie just before we flag a missing final boundary) and see if the final boundary is there (sans whitespace)? If it is, then mark it there and throw a different flag marking a malformed final boundary?

Yes, I agree.

rcbarnett-zz commented 10 years ago

brectanus: You may be able to additionally check RESPONSE_BODY with a regex to see if this is the case instead of ignoring everything, but, it is probably easiest to just disable request body processing (fixed yours slightly - watch those unescaped dots):

SecRule REMOTE_ADDR "^(?:10|127|192.168|172.16)." "chain,t:none" SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:SecRequestBodyAccess=off"

Now, you may also have to use ctl to remove the rule checking for the REQBODY_PROCESSOR_ERROR flag as well (or it will block if set to blocking).

rcbarnett-zz commented 10 years ago

jasonhaar: First off - I think you meant "ctl:RequestBodyAccess=off" - it doesn't like "ctl:SecRequestBodyAccess=off"

Secondly, just doing RequestBodyAccess isn't enough - that's why I suggested disabling the entire engine via "ctl:AuditEngine=off".

I already have the following in place due to previous DocuShare client issues:

           SecRule &REQUEST_HEADERS:DocuShare-Version "@eq 1" "chain,phase:1,t:none,pass,nolog"
            SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:requestBodyAccess=off"

And that doesn't stop the current "Final boundary missing" errors (which do have User-Agent: DsAxess and the Version header). If you think there's a less dramatic option to disabling the entire engine - that will do the job - I'm all ears :-)

Thanks

Jason

rcbarnett-zz commented 10 years ago

jasonhaar:
Gah! Well I went ahead and tried the "AuditEngine=off" option - and it didn't help. I'm already seeing the same errors. So that's weird: I've got a connection from a 10.* address containing the User-Agent: DsAxess/ and yet the engine is not being disabled?

            SecRule REMOTE_ADDR "^(10|127|192\.168|172\.16)\." "chain,t:none"
            SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:AuditEngine=off"

I've just tried ResponseBodyAccess=off (and requestBodyAccess=off) and that didn't work either. What's left? Help? :-)

Jason

rcbarnett-zz commented 10 years ago

brectanus: Yes, sorry, no "sec" prefix for ctl actions. Also, it should have been in phase:1

Are you wanting to just not log (audit)? or not block?

Doing ctl:auditEngine=off will just not audit, but may still block.

Using ctl:requestBodyAccess=off in phase 1 (providing nothing else turns it back on) should prevent the body from being inspected. If not, then it is a bug. Check the debug log (level 9) and make sure this rule fires and that nothing is turning it back on

SecRule REMOTE_ADDR "^(10|127|192.168|172.16)." "chain,t:none,phase:1" SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:requestBodyAccess=off"

rcbarnett-zz commented 10 years ago

jasonhaar:
FYI I enabled the following

SecRule REMOTE_ADDR "^(10|127|192.168|172.16)." "chain,t:none,phase:1" SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:requestBodyAccess=off"

and the "final boundary missing" errors haven't disappeared. I''ll attach a sample to this ticket.

So does mean it's a bug? (2.5.7)

Thanks

Jason

rcbarnett-zz commented 10 years ago

jasonhaar: sample triggering "Final boundary missing" error even with ctl:requestBodyAccess=off

rcbarnett-zz commented 10 years ago

rbarnett: Can you please confirm exactly where you are placing this rule within your configs? Are you specifying it before all other rules?

rcbarnett-zz commented 10 years ago

jasonhaar:
OK - does this answer your question? I don't actually know what order apache loads configs, but hopefully you do. "trl" is our DocuShare server.

shell> cd /etc/httpd/conf.d/ shell> egrep "SecRule" *conf mod_security.conf: #this loads all the modsec settings my-ip.conf: SecRuleEngine On my-ip.conf: SecRuleEngine On WAF-trl.conf: #SecRuleEngine On WAF-trl.conf: SecRule REMOTE_ADDR "^(10|127|192.168|172.16)." "chain,t:none,phase:1" WAF-trl.conf: SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:requestBodyAccess=off" WAF-trl.conf: SecRule &REQUEST_HEADERS:DocuShare-Version "@eq 1" "chain,phase:1,t:none,pass,nolog" WAF-trl.conf: SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:requestBodyAccess=off"

I have removed other WAF-*.conf file matches, however all of them only define modsec rules within VirtualHost areas, so shouldn't impact settings within WAF-trl.conf - which is also within VirtualHost areas.

Hope that's clear

Jason

rcbarnett-zz commented 10 years ago

jasonhaar: Hi there

I haven't seen any further movement on this one, so I was wondering if there was a workaround I could do for this in the interim? Because of this issue (I won't call it a bug as frankly it's Xerox's fault) we have never been able to go "block mode" on our Docushare server. So I was wondering what was the best way to deactivate this check?

I think I'd be best off just disabling modsecurity (actions and logging) all together for internal Docushare clients. We only expect internal users to be using the offending Docushare client anyway. So would the following disable modsec totally for internal traffic from the DsAxess client?

SecRule REMOTE_ADDR "^(10|127|192.168|172.16)." "chain,t:none" SecRule REQUEST_HEADERS:User-Agent "^DsAxess/" "t:none,ctl:SecAuditEngine=off"

Thanks

rcbarnett-zz commented 10 years ago

ivanr: Multipart boundaries are supposed to always start on a new line but, in the attached example, the last boundary doesn't. Unless you've erased the new line by mistake, that Docushare request is invalid. It probably works for them because they control both the client and the server.

I am going to leave this open to check if our parser can be tweaked to parse this properly. We would still have to raise a warning, but at least then you'd be able to create an exception for the case in the rules.

rcbarnett-zz commented 10 years ago

bpinto: Hi Jason,

Did you found a SecRule solution for this problem ? Still having this issue ? Do you have a newer version of DocuShare ?

thanks

Breno