[Profiling x APM Integration] Use both `host.hostname` and `host.name` to correlate profiling data on APM

[cauemarcondes]

Currently, we are only fetching host names based on host.hostname field. We should also run a term aggs on host.name and merge the results before passing it to the Profiling API.

https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/apm/server/routes/profiling/get_service_host_names.ts#L50

There's a comment here on the HOST_HOSTNAME saying we should not use it we should probably look at the places where this field is being used and replace it for HOST_NAME.

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[Danouchka]

+1 in APM we can override host.hostname values. So both values in APM host.name and APM host.hostname should be searched in PROFILING host.name. Thank you

[florianl]

Starting with 8.16 a new option is in place, to correlate APM information with Universal Profiling information. Instead of host.hostname or host.name for the correlation container.id should be used instead. container.id is only available for containerized environments. The assumption is, that is in most environment the case.

With https://github.com/elastic/elasticsearch/pull/111969 the elasticsearch side got merged for Universal Profiling. APM integrations already report container.id, so there is no change.

@smith // @elastic/obs-ux-infra_services-team can you update the kibana side, so that container.id is used for the correlation for the 8.16 release?

[cauemarcondes]

@florianl should we keep the host.name as a fallback in case container.id doesn't exist?

[florianl]

@cauemarcondes - yes, please keep the host.name approach as fallback.