[Observability] Streamline telemetry

[jasonrhodes]

Following from the discussion in #36895, we want to streamline the telemetry we collect across observability apps in Kibana.

Open question(s):

• It's not clear what we want telemetry to track. In infra/logs/apm we currently try to track user behavior in the UI (did a user visit the services page), but mix that with raw values (number of agents, number of logs, etc) -- if we are curious about how many agents are running in a cluster, why not just do an ES query directly in the usage collector's fetch method?
• It looks as though everywhere that we collect data seems to increase the counters endlessly (until a server restart in infra/logs, and forever via saved object in APM) -- so what does the "last_24_hours" data really report right now and what do we want it to do?

AC:

• See above open questions -- if possible we may want to move to direct ES queries in the collector fetch methods
• For Infra and Logs collection:
  • Register two separate usage collectors, one for infra metrics and one for logs, the same way we call registerFeature twice to register them as separate Kibana apps
  • This will let us register the collectors as stack_stats.kibana.plugins.logs and stack_stats.kibana.plugins.infra
  • Change collection period to "last 7 days" instead of "last 24 hours"
  • Change mappings to:
  • stack_stats.kibana.plugins.logs.last_7_days.hits
  • stack_stats.kibana.plugins.infra.last_7_days.hosts.hits
  • stack_stats.kibana.plugins.infra.last_7_days.docker.hits
  • stack_stats.kibana.plugins.infra.last_7_days.kubernetes.hits
  • TBD: Report on calls that return data, not just any call (Jason is looking into this)
• For APM collection:
  • Change collection period to "last 7 days" instead of "last 24 hours"
  • TBD: Report on calls that return data (Jason is looking into this)
• For Uptime collection, make sure we update what's being worked on here -> elastic/kibana#34437 to match what we are doing in this ticket

Notes:

• Telemetry docs: https://github.com/elastic/kibana/blob/master/src/legacy/server/usage/README.md
• Existing mapping can be found in /config/templates/xpack-phone-home.json (telemetry repo)

[elasticmachine]

Pinging @elastic/infra-logs-ui

[tbragin]

@jasonrhodes Also, Uptime telemetry PR should be merged soon: https://github.com/elastic/kibana/pull/34437

It is implemented in a way that matches existing Infra+Logs UI (e.g. 24-hour collection interval), intentionally -- this way it works the same as others until this work happens across the board.

[jasonrhodes]

@tbragin great, I added a note in the ticket description to make sure we update that to match this.

[jasonrhodes]

Closed in favor of #40438