JSON body as nested object in Elasticsearch

[ddrozdov]

The response body is now stored in Elasticsearch as a plain string. In case of monitoring Spring Boot metrics that are returned as a JSON, there is no way to query these metrics in Elasticsearch and visualize them in Kibana. Please provide an option to store the JSON response as some nested structure in the database. There could probably be a parameter in the config file to specify an expected type of the HTTP response: text/json/...

[christiangalsterer]

Thanks for providing the PR. I'm currently travelling with limited bandwidth so please allow some days for reviewing.

[christiangalsterer]

Have just looked a little bit into it and would have the following question. Have you tried to set the codec value in the logstash beats input plugin (https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-codec)? If you said this to json then the body should get rendered as json (not tested).

[ddrozdov]

I use the elasticsearch output for the beat, not the logstash one. The thing is that the event itself is sent to elasticsearch as JSON, but the body field is of string type in the code, so whatever is there in the body is escaped when being serialized to JSON.

[theredcat]

For reference, this is what i'm using in logstash to do this (I've set my document_type to httpbeat_json) :

filter {
  if [type] == "httpbeat_json" {
    json {
      source => "response[body]"
      target => "doc"
      remove_field => [ "response[body]" ]
    }
  }
}