Closed marcinguy closed 4 years ago
cc @wargio WDYT?
Hmm. I would say that the encoding is wrong. You should append a \
for each \
you find.
@wargio take a look at example requests. Could be confusing to see.
First request has zsh escaped "
and second zsh escaped both "
and \
Do you see it?
I append \
for each "
and \
Or am I missing something?
@wargio FYI with GROK parsing it worked
You cannot escape the \
from "
escaping backslash, if that is what you mean
No, what I mean is that you need to backslash each of the \
So /my/Uri/path/
becomes \/my\/uri\/
same for any other special chars.
Ahhh... Seems like you can, but don't have to, per specs:
@wargio Let me know your decision. I can add it. Will be some code more however on possibly each request. I hope GROK parser will still work :) I need it for GROK parser.
@wargio Here is JSON RFC:
"characters that MUST be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)."
Source: https://datatracker.ietf.org/doc/rfc8259/?include_text=1
@wargio Latest changes seem to work correctly. As in first tests:
curl http://127.0.0.1/\"a\?b=\<\>\\
2020/09/03 09:19:50 [error] 25894#0: *3 { "ip":"127.0.0.1","server":"127.0.0.1","uri":"/\"a","vers":"0.56","total_processed":"3","total_blocked":"3","config":"learning","cscore0":"$SQL","score0":"8","cscore1":"$XSS","score1":"24","cscore2":"$TRAVERSAL","score2":"4","zone0":"URL","id0":"1001","var_name0":"","zone1":"ARGS","id1":"1205","var_name1":"b","zone2":"ARGS","id2":"1302","var_name2":"b","zone3":"ARGS","id3":"1303","var_name3":"b" }, client: 127.0.0.1, server: localhost, request: "GET /"a?b=<>\ HTTP/1.1", host: "127.0.0.1"
curl http://127.0.0.1/\"\\a\?b=\<\>\\
2020/09/03 09:19:56 [error] 25894#0: *4 { "ip":"127.0.0.1","server":"127.0.0.1","uri":"/\"\\a","vers":"0.56","total_processed":"4","total_blocked":"4","config":"learning","cscore0":"$SQL","score0":"8","cscore1":"$XSS","score1":"24","cscore2":"$TRAVERSAL","score2":"8","zone0":"URL","id0":"1001","var_name0":"","zone1":"URL","id1":"1205","var_name1":"","zone2":"ARGS","id2":"1205","var_name2":"b","zone3":"ARGS","id3":"1302","var_name3":"b","zone4":"ARGS","id4":"1303","var_name4":"b" }, client: 127.0.0.1, server: localhost, request: "GET /"\a?b=<>\ HTTP/1.1", host: "127.0.0.1"
Should me more optimal, only to call unescaping when needed for \
and/or for "
@wargio Added tests. But somehow the test plan complains on the amount of tests. Do you know how to fix it?
@wargio Fixed test. Tests are green now also
@wargio Changed per your requests. Tests seems green.
My GROK rule had some problems. I realized JSON wasn't properly escaped.
GROK rule:
myParsingrule %{date("yyyy/MM/dd HH:mm:ss"):connect_date} \[%{word:status}\] %{number:id}\#%{number:id2}: \*%{number:value} %{data::json}
I think only those needs to be escaped.
Examples, how it now works correctly:
Please review, maybe it can be done better.
The alternative would be to use JSON library, but this would add a dependency, also not sure about the speed.
Thanks,