[Agent Health Dashboards] Zero Integrations Enabled

[jamiehynds]

A bug has been identified, whereby the Overview page on the agent health dashboards is incorrectly reporting 0 integrations enabled, even if several integrations are enabled and ingesting data. Need to determine the root cause and update the dashboard to ensure the integrations enabled count is accurate.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[ebeahan]

I took a look, and it appears this visualization started only reporting zero in 8.9.

I noticed differences in running a cardinality agg in 8.8.2 vs 8.10.1. When I dug in a bit more, it seemed specific to constant_keyword fields. Note the aggregations.0.value of 0 on 8.10.1:

8.8.2

GET logs-*,metrics-*/_search
{
  "aggs": {
    "0": {
      "cardinality": {
        "field": "data_stream.dataset"
      }
    }
  },
  "size": 0
}

{
  "took": 2384,
  "timed_out": false,
  "_shards": {
    "total": 18,
    "successful": 18,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 3314,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "0": {
      "value": 15 <= expected
    }
  }
}

8.10.1

GET logs-*,metrics-*/_search
{
  "aggs": {
    "0": {
      "cardinality": {
        "field": "data_stream.dataset"
      }
    }
  },
  "size": 0
}

{
  "took": 34,
  "timed_out": false,
  "_shards": {
    "total": 166,
    "successful": 166,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "0": {
      "value": 0 <= not expected
    }
  }
}

[ebeahan]

Unrelated observation: does the value for Integrations Enabled need the decimal?

[ebeahan]

Opened ES issue with above finding: https://github.com/elastic/elasticsearch/issues/99776

[jamiehynds]

Thanks for taking a look @ebeahan. With regards to the decimals, this is a known bug which @P1llus has raised with Kibana and they are working on a fix.

[ebeahan]

ES team added a fix for the cardinality agg issue: https://github.com/elastic/elasticsearch/pull/99814. The fix ships in 8.10.3 and 8.11.0.

[jamiehynds]

Closing as a fix has been available since 8.10.3.