search

eBay / nvidiagpubeat

nvidiagpubeat is an elastic beat that uses NVIDIA System Management Interface (nvidia-smi) to monitor NVIDIA GPU devices and can ingest metrics into Elastic search cluster, with support for both 6.x and 7.x versions of beats. nvidia-smi is a command line utility, based on top of the NVIDIA Management Library (NVML), intended to aid in the management and monitoring of NVIDIA GPU devices.
https://github.com/eBay/nvidiagpubeat
Apache License 2.0
54 stars 22 forks source link

Field @timestamp is mapped as type:string #15

Closed ml-jzimmermann closed 5 years ago

ml-jzimmermann commented 5 years ago

The timestamp field is mapped as string. Please update the mapping before the index creation to fix this issue.

This issue was created as a result of the #11 Issue.

Thanks, Julius

deepujain commented 5 years ago

Hello Julius I do not have the full setup with me. Upon running nvidiagpubeat locally with localnvidiasmi (simulator to generate dummy gpu utilization). All I see is

{
  "@timestamp": "2019-03-25T22:58:35.872Z",
....
}

Can you elaborate, how the field is being mapped as string. I believe this field is coming from line 81 from nvidiagpubeat.go and i see the type as Time and not string.

Timestamp: time.Now(),

Can you elaborate, on how to reproduce this issue or provide a PR ?

Regards, Deepak

ml-jzimmermann commented 5 years ago

Hi, I added an IndexPattern "nvidiagpubeat-*" to match the elasticsearch index. The elasticsearch index that is created by the beater has the @timestamp field mapped as "keyword".

https://ibb.co/tQ7yjPg

Regards, Julius

ml-jzimmermann commented 5 years ago

Hi again,

i am closing this issue for now because logstash also fixes this. I dont see / know how to change this behaviour in the code. I suppose to use logstash for the future anyways. This is my logstash config to parse this beater output into elasticsearch:

input { beats { port => "5044" } }

filter { }

output { elasticsearch { hosts => [ "localhost:9200" ] } }

Regards, Julius

nperron commented 3 years ago

Hello, i had the same issue. The timestamp was set as "keyword" type because of wrong template.

[...] "dynamic_templates": { "string_as_keyword" : { "mapping" : { "ignore_above": 1024, "type": "keyword" [...]

In fact, the issue was that the templates files were absent (by default, should be present in the same folder than the binary), so nvidiagpubeat wasn't able to set it correctly to ElasticSearch.