search

christiangalsterer / httpbeat

Elastic Beat to call HTTP endpoints
Apache License 2.0
107 stars 40 forks source link

Problem in parsing Spring Boot Metrics #16

Closed udikidron closed 7 years ago

udikidron commented 7 years ago

Hi,

I'm trying to use 'httpbeat' in order to sample a spring boot application mertics. The url I'm sampling is: http://localhost:8080/metrics The output format is of type: 'json'

The response contains the following JSON:

{"mem":824349,"mem.free":222040,"processors":8,"instance.uptime":11355545,"uptime":9690309,"systemload.average":4.31494140625,"heap.committed":748032,"heap.init":262144,"heap.used":525991,"heap":3728384,"nonheap.committed":78144,"nonheap.init":2496,"nonheap.used":76320,"nonheap":0,"threads.peak":56,"threads.daemon":33,"threads.totalStarted":1398,"threads":37,"classes":8172,"classes.loaded":8242,"classes.unloaded":70,"gc.ps_scavenge.count":14,"gc.ps_scavenge.time":132,"gc.ps_marksweep.count":4,"gc.ps_marksweep.time":612,"counter.status.200.health":343,"counter.status.200.metrics.root":2,"counter.status.200.metrics.name:.-star-":107,"gauge.response.star-star.favicon.ico":19.0,"counter.status.200.star-star.favicon.ico":1,"gauge.response.metrics":2.0,"gauge.response.metrics.root":2.0,"gauge.response.metrics.name:.-star-":1.0,"gauge.response.health":2.0,"counter.status.200.metrics":879,"httpsessions.max":-1,"httpsessions.active":0}

You can see that some of the values in the response contain a dot '.'

While 'httpbeat' tring to send the response to elasticSearch I got the following error:

2017/01/19 16:41:36.714328 client.go:432: WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"Field name [mem.free] cannot contain '.'"}

I assume that I can change the mapping in httpbeat.template.json to make elasticsearch tollerant for this naming confession.

Is it true? If yes - can you please advise how should I change the mapping?

Thanks!

christiangalsterer commented 7 years ago

One way is to set the son_dot_mode and set the value to replace as an additional parameter for the httpbeat, see docs/configuration.asciidoc for an example.

Note: While testing I found a bug when setting the value to 'unflatten'.

christiangalsterer commented 7 years ago

There is an interesting discussion on dots in field names in: https://github.com/elastic/elasticsearch/issues/15951

udikidron commented 7 years ago

I took your suggestion, json_dot_mode: replace And it solved my problem. Thanks.