search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.15k stars 4.91k forks source link

Couchbase module throws json parse exception on basicStats.opsPerSec #18949

Closed amrishraje closed 3 years ago

amrishraje commented 4 years ago

Please refer to below post: https://discuss.elastic.co/t/couchbase-module-throws-json-parse-exception-on-basicstats-opspersec/235455

Issue: Metricbeat Couchbase module throws the below error Error: %!(EXTRA *json.UnmarshalTypeError=json: cannot unmarshal number 790.2097902097902 into Go struct field BucketBasicStats.opsPerSec of type int64)

Issue is in the data.go file. The code makes an assumption that couchbase API pools/default/buckets returns an int where as it actually returns a float. I have checked this with version 5.x and 6.x of couchbase.

I see this issue with metricbeat version 6.8.3 (amd64), libbeat 6.8.3 running on RHEL 7.7

Error Details: 

2020-06-02T22:19:20.396-0500    DEBUG   [cfgfile]       cfgfile/reload.go:195   Number of module configs found: 1
2020-06-02T22:19:20.396-0500    DEBUG   [reload]        cfgfile/list.go:62      Starting reload procedure, current runners: 0
2020-06-02T22:19:20.396-0500    DEBUG   [reload]        cfgfile/list.go:80      Start list: 1, Stop list: 0
2020-06-02T22:19:20.396-0500    DEBUG   [processors]    processors/processor.go:66      Processors:
2020-06-02T22:19:20.396-0500    DEBUG   [reload]        cfgfile/list.go:101     Starting runner: couchbase [metricsets=1]
2020-06-02T22:19:20.396-0500    DEBUG   [module]        module/wrapper.go:117   Starting Wrapper[name=couchbase, len(metricSetWrappers)=1]
2020-06-02T22:19:20.396-0500    INFO    cfgfile/reload.go:205   Loading of config files completed.
2020-06-02T22:19:20.396-0500    DEBUG   [module]        module/wrapper.go:179   Starting metricSetWrapper[module=couchbase, name=bucket, host=localhost:8091]
2020-06-02T22:19:20.425-0500    ERROR   bucket/data.go:53       Error: %!(EXTRA *json.UnmarshalTypeError=json: cannot unmarshal number 790.2097902097902 into Go struct field BucketBasicStats.opsPerSec of type int64)
2020-06-02T22:19:20.425-0500    DEBUG   [publish]       pipeline/processor.go:309       Publish event: {
  "@timestamp": "2020-06-03T03:19:20.396Z",
  "@metadata": {
    "beat": "metricbeat",
amrishraje commented 4 years ago

I also worked with couchbase to get their position on what their /pools/default/buckets API is returning. Their response is documented below. Based on JSON spec, the API is returning a numeric type which can be both an int or a float. We should not interpret it as an int64 in the metricbeat couchbase module.

From the initial review of your request, please find our below answers - The issue is that couchbase returns the basicStats.opsPerSec field as a float instead of int. Couchbase returns the basicStats.opsPerSec field as numeric type which is in JSON schema. As per JSON Schema documentation, the type number can either be integers or floating point numbers. Also in the provided documentation, we would like to let you know that we do not explicitly see that the return value is of type intonly. Hence we would like to re-iterate that the return value should be interpreted as numerictype which can be either integers or floating point numbers. We hope this answers your request. Please let us know if you have further questions regarding this and we're glad to help.

elasticmachine commented 4 years ago

Pinging @elastic/integrations-services (Team:Services)

BlakeWills commented 4 years ago

The referenced commit makes this look like it's fixed in 7.8, but I've just tried with that version and we get the same issues.

ERROR   node/data.go:80 Error: json: cannot unmarshal number 2.002002002002002 into Go struct field NodeInterestingStats.nodes.interestingStats.cmd_get of type int64

ERROR   bucket/data.go:53   Error: json: cannot unmarshal number 10.004004004004 into Go struct field BucketBasicStats.basicStats.opsPerSec of type int64
sayden commented 4 years ago

I can confirm, we fix a different field but we left this one. It's not ideal to parse everything as double in Elasticsearch (it uses more space at the end) but the fact that JSON don't differentiate their numeric type between integer or floating point will force Beats to assume that everything can be potentially a double type.

nyetwurk commented 4 years ago

Not sure if this is related, but using logstash with filebeat/metricbeat consistently results in "0" being interpreted as an int.

I've even seen "0.0" get converted to "0" int, until a new index rolls around and the value is a non-zero non-integer.

This is for schemaless data with no templates in ES. The root cause is most certainly JSON not distinguishing float from int in Numeric.

sayden commented 3 years ago

Fixed in https://github.com/elastic/beats/pull/19055