search

lucashenning / logstash-filter-rest

REST Filter for Logstash
Other
43 stars 51 forks source link

Some values types get transformed to string #24

Open rickys13 opened 7 years ago

rickys13 commented 7 years ago

This question was originally posted inside the comments for issue #22

Is there a way to prevent stringifying the values?

if I set: body => { offset => null pageSize => 1 saySomething => "hello" preferMethod => false }

I get in my POST service request body: { "offset": "null", "pageSize": 1, "saySomething": "hello", "preferMethod": "false" }

only the integer was left alone but the boolean and null were transformed to strings.

gandalfb commented 7 years ago

Hi @rickys13

this is related to LogStash::Json.load which is used to parse the response body as json into ruby. Parsing the json is using https://github.com/guyboertje/jrjackson as far as I can tell.

Options can be set within the LogStash::Json.load call, but I do not see a variable, which may suite your needs.

A workaround might be the mutate filter: https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-convert From code, those types are supported: %w(string integer float boolean) (ref: https://github.com/logstash-plugins/logstash-filter-mutate/blob/master/lib/logstash/filters/mutate.rb)

rickys13 commented 7 years ago

@gandalfb I am probably not quite understanding your suggestion. I am looking for avoiding stringification of the values as they are sent in the request body, your suggestion of using mutate-convert wouldn't do that right? I think that would be applicable to modify the response right?

gandalfb commented 7 years ago

I got this wrong, sorry.

Then we are with LogStash::Json.dump, as json goes as text over the wire. Here JrJackson is used as well to serialize the json text.

Code, of the what to what map: https://github.com/guyboertje/jrjackson/blob/master/lib/jrjackson/jrjackson.rb#L61

And in your config: ruby uses for 'null' the 'nil'. In this case it doesn't matter, as being mapped to a "null" string.

But in general a consuming REST should handle such things, once creating the object (C++, Java, ...) out of the transmitted json text. And if you request 'nil', it is obsolete/optional anyway, isn't it?

As those Functions are broadly used throughout various Logstash plugins and core, I do not see a major issue here. Maybe this is something for jrjackson?