Telemetry V2 Migration of Existing Stack Stats

[dominiqueclarke]

This issue is targeting migration our current kibana-snapshot-powered telemetry to the new ingestion system. Known tasks (based on the end-to-end example) are:

• [x] Add index configuration https://github.com/elastic/telemetry/pull/970
• [x] Add staging jenkins indexing job https://github.com/elastic/telemetry/pull/970

An example from the APM team further helps in gaining a sense of the necessary tasks https://github.com/elastic/telemetry/pull/798/files

[dominiqueclarke]

@paulb-elastic This is complete with elastic/telemetry#970.

One thing to note is that the telemetry can be found in https://telemetry-v2-staging.elastic.dev/, not https://stack-telemetry.elastic.com. The legacy stack stats telemetry data is only forwarded to the staging cluster, not prod. It's a bit confusing, because the data that is being forwarded is production customer data. Nevertheless, we can view this data, for production users, at https://telemetry-v2-staging.elastic.dev.

Please note, I have not created any dashboards, but I have verified the existence of the data at https://telemetry-v2-staging.elastic.dev/. It seems like as a team, we haven't been using stack stats much lately, so it's unclear which dashboards would be valuable. However, we now have the data available in telemetry-v2 format should we need to update it or access it for visualizations down the line.

[paulb-elastic]

Thanks @dominiqueclarke

@drewpost if there are specific dashboards you need creating, can you create an issue please?