[Infra UI] Exclude containers from Host dashboards

[rogercoll]

Kibana version: 8.14.3

The Host dashboards rely on the host.name attribute in order to show and count the number of monitored hosts. In OpenTelemetry, SDKs services running in containerized environment report this attribute too and Kibana will identify them and create host entities but without any related metric:

For the previous screenshot, only 3 out of the 11 identified hosts are actual computing instances (K8s nodes). The remaining "hosts" are APM agents running in a container environment and providing the host.name attribute in their signals.

The main issue is that the host attributes are enabled by default in many SDKs (especially in Elatic's distributions). I would opt for removing this attribute if the SDK detects that is running in a containerized environment, but that would require agreement in the Otel community + extra logic in all the SDKs.

Would it be an option to modify the default dashboards to discard any "host" that also has "container" attributes?

OpenTelemetry host attributes: https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-attributes

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[crespocarlos]

Explanation of why this happens: https://github.com/elastic/opentelemetry-dev/issues/365#issuecomment-2285524668

[crespocarlos]

Closing this in favor of #190418. The UI will discard any document that doesn't have event.module: system or doesn't come from APM. If a host comes from APM and doesn't have metrics, the UI will provide a way for users to troubleshoot the issue and onboard system metrics for this host.