search

dell / iDRAC-Telemetry-Reference-Tools

Reference toolset for PowerEdge telemetry metric collection and integration with analytics and visualization solutions.
https://github.com/dell/iDRAC-Telemetry-Reference-Tools
Apache License 2.0
28 stars 20 forks source link

Elastic Pipeline missing documents #55

Closed 00-eight closed 1 year ago

00-eight commented 2 years ago

When using the docker-compose elk reference pipeline there are gaps between sensor readings. kibana_db_gaps Viewing Aggregation from ES it is clear that documents are missing from these sensors. 

  "aggregations": {
    "by_context": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "CPU2 Temp",
          "doc_count": 13520
        },
        {
          "key": "CPU1 Temp",
          "doc_count": 8088
        }
      ]
    }
  }

When viewing a parallel SSE from iDRAC it is clear all readings are present. When viewing es-ingester logs it is clear that all readings are being read from databus 

/* https://github.com/dell/iDRAC-Telemetry-Reference-Tools/blob/main/cmd/elkpump-basic/elkpump-basic.go */
            log.Print("value: ", value) // line 80

I suspect that these readings are getting lost during handleGroups call to esapi.bulk

Corresponding es-ingester logs and ES aggregation for dashboard image es-ingester_and_kibana_agg.log

SailM23 commented 2 years ago

Thanks for opening this ticket, will look into the issue and get back...

SailM23 commented 1 year ago

This is issue is closed as the data discrepancy between the Fan1,Fan2 etc of idrac SSE is accumulated and showing as big difference in the doc_count of the sql query in the Kibana. More details in the Jit story TLM-573.