[Metricbeat] Add Prometheus module remote_read capabilities

[fred-maussion]

Describe the enhancement: Today the Elastic stack is interacting with Prometheus in a “one way” mode only. Through the Prometheus module , metricbeat allow Elastic users to write Prometheus data into Elasticsearch. Those data are available to be used in Kibana. To implement the writing of Prometheus data, Metricbeat implements half of the Prometheus storage API, the “remote write” part. Because Metricbeat implements only half of the API, Prometheus server cannot “read back” the data stored in Elasticsearch. And thus do not qualify as a remote storage for Prometheus.

Describe a specific use case for the enhancement or feature: We would like metricbeat to implement the remote read part of Prometheus API. Then, Prometheus could use Elasticsearch as a remote store. Samples of such implementation exists. Similar tor the “write” endpoint metricbeat would expose a “read” endpoint :

- module: prometheus
  metricsets: ["remote_read"]
  host: "localhost"
  port: "9201"

Benefits:

• Migration project from Prometheus to Elastic on huge and complex environment can be tough and providing a smooth way to migrate the backend without touching the ingestion workflow is a significant value add to complex project as there is no disruption in the overall query workflow process.
• It will also minimize the end-user change management.
• Present Elastic a full compatible backend for Prometheus

[botelastic[bot]]

This issue doesn't have a Team:<team> label.

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[dragonsashimi]

👍