Open landergate opened 7 years ago
csanders-git
commented
7 years ago Looks to be a libinjection rule. interesting, not quite sure why this is, maybe it wasnt done with other langs in mind. I know modsec's regex engine also has problems with Cyrillic chars just cause of PCRE
dune73
commented
7 years ago Thank you very much for your report @landergate. This is an upstream problem but very glad you reported. I am going to forward this to the libinjection maintainer asap.
lifeforms
commented
7 years ago Wow, very good find. There are three places where this can go wrong: CRS, ModSecurity, or libinjection.
At first sight, it seems like a false positive in libinjection. But if I feed this input to libinjection itself, in encoded or decoded form, I am not getting a hit:
| Payload | Result |
|---|---|
dor |
not sqli |
дор |
not sqli |
%D0%B4%D0%BE%D1%80 |
not sqli |
So I am not sure it's a libinjection problem. There seems to be something interesting going on in ModSec or CRS. We should narrow the issue first by removing some transformations.
dune73
commented
7 years ago You suspected correctly, @lifeforms. Here is my reduced rule to trigger this FP:
Minimal Rule:
SecRule ARGS:login "@detectSQLi" \
"msg:'SQL Injection Attack Detected via libinjection',\
id:942100,\
phase:request,\
deny,\
t:none,t:utf8toUnicode,t:urlDecodeUni"
Payload:
curl localhost -d "login=дор"
Debug Log:
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][5] Adding request argument (BODY): name "login", value "\xd0\xb4\xd0\xbe\xd1\x80"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Input filter: Completed receiving request body (length 12).
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Starting phase REQUEST_BODY.
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] This phase consists of 1 rule(s).
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Recipe: Invoking rule 55dcf0641518; [file "/apache/conf/httpd.conf_pod_2017-06-06_12:52"] [line "159"] [id "942100"] [rev "1"].
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][5] Rule 55dcf0641518: SecRule "ARGS:login" "@detectSQLi " "phase:request,log,msg:'SQL Injection Attack Detected via libinjection',id:942100,severity:CRITICAL,rev:1,ver:OWASP_CRS/3.0.0,maturity:1,accuracy:8,block,t:none,t:utf8toUnicode,t:urlDecodeUni,capture,logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:application-multi,tag:language-multi,tag:platform-multi,tag:attack-sqli,tag:OWASP_CRS/WEB_ATTACK/SQL_INJECTION,tag:WASCTC/WASC-19,tag:OWASP_TOP_10/A1,tag:OWASP_AppSensor/CIE1,tag:PCI/6.5.2,setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.sql_injection_score=+%{tx.critical_anomaly_score},setvar:tx.msg=%{rule.msg},setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{matched_var}"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] T (0) urlDecodeUni: "4>@"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Transformation completed in 22 usec.
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Executing operator "detectSQLi" with param "" against ARGS:login.
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] Target value: "4>@"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] Added phrase match to TX.0: 1ov
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] ISSQL: libinjection fingerprint '1ov' matched input '4>@'
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Operator completed in 30 usec.So it looks like a combination of payload and two transformation leads to an internal value that sets off libinjection.
lifeforms
commented
7 years ago T (0) Utf8toUnicode: "%u0434%u043e%u0440"
T (0) urlDecodeUni: "4>@"lol wat
dune73
commented
7 years ago Yes, looks a bit simplistic. Bad implementation in ModSec?
csanders-git
commented
7 years ago @zimmerle can you shed some light on this problem? I know there are tickets in modsec's repo also can we link them here if they apply?
zimmerle
commented
7 years ago Hi @csanders-git, please go ahead and make the links ;)
I will also report something here. Most likely I will look into this tomorrow.
@all Thanks for the report ;)
csanders-git
commented
7 years ago 
victorhora
commented
7 years ago Hey guys, it looks like we have a few problems here with this issue. First lets see if ModSecurity is handling these transformations right:
[07/Jun/2017:09:47:35 --0400] [localhost/sid#7f9c93b86418][rid#7f9c93acb0a0][/][5] Adding request argument (BODY): name "login", value "\xd0\xb4\xd0\xbe\xd1\x80"\xd0\xb4\xd0\xbe\xd1\x80 looks correct UTF-8 encoding of дор.
[07/Jun/2017:08:46:35 --0400] [localhost/sid#7f138e17bd20][rid#7f138e0bd0a0][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"0x0434 = CYRILLIC SMALL LETTER DE (aka "De") 0x043E = "о" CYRILLIC SMALL LETTER O 0x0440 = "р" CYRILLIC SMALL LETTER ER (aka "Er")
This looks right as well.
Moving on, there's the SecUnicodeMapFile which t:urlDecodeUni uses to map Unicode for normalization and we have on the recommend modsecurity.conf file which by default points to Code point 20127 US-ASCII mapping.
If you simply try to change the code point to 1251 (aka Windows-1251) it still won't work as for some reason these cyrillic characters are missing from the mapping file. But I believe they should be there according to unicode.org.
But if you add the missing Unicode characters and their corresponding ASCII codes (0434:64 043e:6f 0440:70) to the code point the rule works fine with the input:
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Recipe: Invoking rule 7efd043ded48; [file "/etc/apache2/conf-enabled/modsecurity.conf"] [line "258"] [id "942102"].
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][5] Rule 7efd043ded48: SecRule "ARGS:login" "@detectSQLi " "phase:request,log,auditlog,msg:'SQL Injection Attack Detected via libinjection',id:942102,deny,t:none,t:utf8toUnicode,t:urlDecodeUni"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] T (0) urlDecodeUni: "dop"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Transformation completed in 12 usec.
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Executing operator "detectSQLi" with param "" against ARGS:login.
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] Target value: "dop"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] ISSQL: not sqli, no libinjection sqli fingerprint matched input 'dop'
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Operator completed in 17 usec.
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Rule returned 0.So I think the the transformations are working fine, but I think we need to do some adjustments to the mapping file or come up with a different solution altogether.
Interestingly enough looks like that other software that fails to recognize these characters does the same thing as ModSecurity. From Burp:
csanders-git
commented
7 years ago https://github.com/SpiderLabs/ModSecurity/issues/348#issuecomment-307545146 please see the very thoughtful comment @marcstern left on this other thread.
landergate
commented
7 years ago Any possible workaround approach at this moment?
Like allow exact \x183>@L data input and not disabling rule completely?
dune73
commented
7 years ago @landergate : This does not look like a quick fix and I think there is no way around disabling the rule for the login parameter for the time being. Of course, if you know the acceptable values that trigger false positives, then you can also disable the rule depending on the value of login.
lifeforms
commented
7 years ago Victor has some good analysis on this, if expanding the mapping would solve this then it would be a real constructive solution... I'm not familiar with that part of ModSec myself. Tagging this for 3.1.0 as the problem happen on multiple rules and we need to support our Russian friends!
nemphys
commented
5 years ago I am also facing false positives caused by rule 942100 (running ModSecurity v3/master and CRS 3.1) on post params in greek (utf8). The question mark (?) in greek is ";" (semicolon) so I guess this makes it easy to become trigger-happy more often.
After tracking down a false positive, I came to the conclusion that if an apostrophe (') and a greek question mark (semicolon, ";") appears on the same post argument, the rule triggers.
Since the use of those 2 characters is quite common in greek, I would appreciate any suggestions.
PS. I have already tried experimenting with different code points in unicode.mapping (started with 1253 greek but also tried 850, since it seems to include more greek unicode characters), but nothing changed.
AirisX
commented
5 years ago Hey guys, it looks like we have a few problems here with this issue. First lets see if ModSecurity is handling these transformations right:
[07/Jun/2017:09:47:35 --0400] [localhost/sid#7f9c93b86418][rid#7f9c93acb0a0][/][5] Adding request argument (BODY): name "login", value "\xd0\xb4\xd0\xbe\xd1\x80"\xd0\xb4\xd0\xbe\xd1\x80 looks correct UTF-8 encoding of дор.
[07/Jun/2017:08:46:35 --0400] [localhost/sid#7f138e17bd20][rid#7f138e0bd0a0][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"0x0434 = CYRILLIC SMALL LETTER DE (aka "De") 0x043E = "о" CYRILLIC SMALL LETTER O 0x0440 = "р" CYRILLIC SMALL LETTER ER (aka "Er")
This looks right as well.
Moving on, there's the SecUnicodeMapFile which t:urlDecodeUni uses to map Unicode for normalization and we have on the recommend modsecurity.conf file which by default points to Code point 20127 US-ASCII mapping.
If you simply try to change the code point to 1251 (aka Windows-1251) it still won't work as for some reason these cyrillic characters are missing from the mapping file. But I believe they should be there according to unicode.org.
But if you add the missing Unicode characters and their corresponding ASCII codes (0434:64 043e:6f 0440:70) to the code point the rule works fine with the input:
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Recipe: Invoking rule 7efd043ded48; [file "/etc/apache2/conf-enabled/modsecurity.conf"] [line "258"] [id "942102"]. [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][5] Rule 7efd043ded48: SecRule "ARGS:login" "@detectSQLi " "phase:request,log,auditlog,msg:'SQL Injection Attack Detected via libinjection',id:942102,deny,t:none,t:utf8toUnicode,t:urlDecodeUni" [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440" [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] T (0) urlDecodeUni: "dop" [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Transformation completed in 12 usec. [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Executing operator "detectSQLi" with param "" against ARGS:login. [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] Target value: "dop" [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] ISSQL: not sqli, no libinjection sqli fingerprint matched input 'dop' [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Operator completed in 17 usec. [07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Rule returned 0.So I think the the transformations are working fine, but I think we need to do some adjustments to the mapping file or come up with a different solution altogether.
Interestingly enough looks like that other software that fails to recognize these characters does the same thing as ModSecurity. From Burp:
Hi @victorhora ,
in your example you made mapping of the Cyrillic characters on their analogue in the Latin alphabet. E.g.:
0x0434 -> d (0x64 in ASCII)
0x043E -> o (0x6f in ASCII)
0x0440 -> p (0x70 in ASCII)Although, for mentioned code page 1251 (extension of the ASCII), correct mapping is:
0x0434 -> д (0xe4 in 1251)
0x043E -> о (0xee in 1251)
0x0440 -> р (0xf0 in 1251)The question is - do we always need make mapping for non latin symbols to their latin analogue to get working solution? Can you explain?
AirisX
commented
5 years ago Some words about CP1251 mapping.
Unicode table contains a special range for Cyrillic characters. This is 0400 - 04FF (https://unicode-table.com/en/blocks/cyrillic/). They are used in many languages, the writing of which is based on them. But as for the Russian language, this range is much narrower: 0410 - 044F and two characters outside this range are 0401 (letter "Ё") and 0451 (small letter "ё"). Thus, I think the correct filling of CP1251 mapping looks like this:
0410:c0 0411:c1 0412:c2 0413:c3 0414:c4 0415:c5 0416:c6 0417:c7 0418:c8 0419:c9 041a:ca 041b:cb 041c:cc 041d:cd 041e:ce 041f:cf 0420:d0 0421:d1 0422:d2 0423:d3 0424:d4 0425:d5 0426:d6 0427:d7 0428:d8 0429:d9 042a:da 042b:db 042c:dc 042d:dd 042e:de 042f:df 0430:e0 0431:e1 0432:e2 0433:e3 0434:e4 0435:e5 0436:e6 0437:e7 0438:e8 0439:e9 043a:ea 043b:eb 043c:ec 043d:ed 043e:ee 043f:ef 0440:f0 0441:f1 0442:f2 0443:f3 0444:f4 0445:f5 0446:f6 0447:f7 0448:f8 0449:f9 044a:fa 044b:fb 044c:fc 044d:fd 044e:fe 044f:ff 0401:a8 0451:b8
Using this mapping for payload with russian characters will not lead to a false positive in example with "Игорь".
But code page 1251 also contains characters of the Ukrainian, Bulgarian and Belarusian alphabets. Such as ї, Ї, і, І and so on (see http://ascii-table.com/codepage.php?1251). They also need to be added to the mapping.
thevulnhunter
commented
5 years ago What to do for this false positive from ModSecurity? Message: Warning. detected SQLi using libinjection with fingerprint 'novc' [file "/etc/apache2/modsecurity.d/owasp-modsecurity/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "66"] [id "942100"] [msg "SQL Injection Attack Detected via libinjection"] [data "Matched Data: novc found within ARGS:password:
lifeforms
commented
5 years ago @jibz08 That's an issue unrelated to this thread. Generally you want to exclude a password field from CRS checking. This is normally done by adding to your ModSec config a rule like:
SecRule REQUEST_URI "@streq /your/page/uri" \
"id:12345,phase:1,t:none,nolog,pass,\
ctl:ruleRemoveTargetByTag=.*;ARGS:password"@jibz08 That's an issue unrelated to this thread. Generally you want to exclude a password field from CRS checking. This is normally done by adding to your ModSec config a rule like:
SecRule REQUEST_URI "@streq /your/page/uri" \ "id:12345,phase:1,t:none,nolog,pass,\ ctl:ruleRemoveTargetByTag=.*;ARGS:password"
Thanks @lifeforms ,it helped .
github-actions[bot]
commented
4 years ago This issue has been open 120 days with no activity. Remove the stale label or comment, or this will be closed in 14 days
dune73
commented
4 years ago Decision during the CRS project chat on March 2, 2020: @dune73 will get in touch with the libinjection project to try and get things moving again.
https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/1683#issuecomment-593584538
Any thought on why two cyrillic symbols "ор" could trigger this rule? It happens in different conditions near other symbols, but could not happen at all with other symbols.
And what would be the best approach without omitting rules at all?
login=Игорь
It's a legit cyrillic first name, pronounced as "Igor".
login=дор
Just experimenting with "ор" combination.
login=ббббббор
login=Симафор