Closed andsel closed 1 year ago
This is expected behavior. The slash you’re using is escaping the doublequote inside of the doubleqoute encapsulated string. Try using 3 \ characters (one more to escape the slash and another to escape the quote).
@MikeKemmerer I would agree with you if the generator
used the the double quotes:
message => "{\"Field1\": \"22222wwwww\"www\"wwwwww\"}"
but however to have to be right it should have quoted also the quote character like in:
message => "{\"Field1\": \"22222wwwww\\\"www\\\"wwwwww\"}"
but is uses the single quote:
message => '{"Field1": "22222wwwww\"www\"wwwwww"}'
Replacing the HTTP filter with HTTP output
output {
http {
ecs_compatibility => "disabled"
http_method => "post"
url => "http://localhost:8080"
}
}
Behaves like expected on WireMock
Connection: [keep-alive]
Content-Type: [application/json]
Content-Length: [206]
Host: [localhost:8080]
User-Agent: [Manticore 0.9.1]
Accept-Encoding: [gzip,deflate]
{"Field1":"22222wwwww\"www\"wwwwww","@version":"1","event":{"original":"{\"Field1\": \"22222wwwww\\\"www\\\"wwwwww\"}","sequence":0},"host":{"name":"kalimera"},"@timestamp":"2023-02-28T15:25:13.743720606Z"}
@MikeKemmerer you are right the json
{"Field1": "22222wwwww\"www\"wwwwww"}
is a valid json, on which the Field1
contains the string with a couple of double quotes, and those double quotes needs to be quoted to be represented in json: 22222wwwww"www"wwwwww
Logstash information:
Please include the following information:
bin/logstash --version
)8.5.1
but also tested with7.17.8
JVM (e.g.
java -version
):If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:
java -version
)JAVA_HOME
environment variable if set.OS version (
uname -a
if on a Unix-like system): anyDescription of the problem including expected versus actual behavior: When an field of the event contains quoted characters, like
\"
the payload sent to the server should also contains the same quoting.Steps to reproduce:
Please include a minimal but complete recreation of the problem, including (e.g.) pipeline definition(s), settings, locale, etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.
java -jar wiremock-jre8-standalone-2.35.0.jar --verbose
input { generator { count => 1 message => '{"Field1": "22222wwwww\"www\"wwwwww"}' codec => "json" } }
filter { http { target_body => "response_body" verb => "POST" url => "http://localhost:8080" body => "%{Field1}" } }
output { stdout{ codec => rubydebug } }
22222wwwww"www"wwwwww
22222wwwww\"www\"wwwwww
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> POST / HTTP/1.1 [2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Connection: Keep-Alive [2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> content-type: text/plain [2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Content-Length: 21 [2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Host: localhost:8080 [2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> User-Agent: Manticore 0.9.1 [2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Accept-Encoding: gzip,deflate [2023-02-28T16:09:20,469][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "POST / HTTP/1.1[\r][\n]" [2023-02-28T16:09:20,469][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "content-type: text/plain[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Content-Length: 21[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Host: localhost:8080[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "User-Agent: Manticore 0.9.1[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "[\r][\n]" [2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "22222wwwww"www"wwwwww" [2023-02-28T16:09:20,475][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "HTTP/1.1 404 Not Found[\r][\n]" [2023-02-28T16:09:20,475][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "Content-Type: text/plain[\r][\n]" [2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "Transfer-Encoding: chunked[\r][\n]" [2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "[\r][\n]" [2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "54[\r][\n]" [2023-02-28T16:09:20,477][DEBUG][org.apache.http.headers ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << HTTP/1.1 404 Not Found
2023-02-28 16:09:20.472 Request received: 127.0.0.1 - POST /
Connection: [keep-alive] Content-Type: [text/plain] Content-Length: [21] Host: [localhost:8080] User-Agent: [Manticore 0.9.1] Accept-Encoding: [gzip,deflate] 22222wwwww"www"wwwwww
Matched response definition: (no response definition configured)
Response: HTTP/1.1 404 (no headers)