Add prometheus-exporter-plugin to Opensearch builtin plugins

saeed-mcu commented 1 year ago

Currently, it is not possible to monitor Opensearch performance from the Opensearch dashboard, while Elasticsearch can be monitored through Stack Monitoring easily.

If the prometheus-exporter-plugin is added to the Built-in plugins , monitoring of opensearch will be easy, and it can be have dashboard something like stack monitoring or monitored through Prometheus externally.

lukas-vlcek commented 1 year ago

Hi,

OpenSearch Community projects page lists Prometheus exporter plugin which can be installed very easily (like any other OpenSearch plugin). On top of that the Prometheus exporter plugin repo has a branch mixin which contains Grafana dashboard and Prometheus Alerts (if you do not want to build the dashboard from sources you can download its release here: https://github.com/aiven/prometheus-exporter-plugin-for-opensearch/releases/tag/mixin-1.0.0-rc.1).

Is that what you are looking for?

Regards, Lukáš

peternied commented 8 months ago

This seems like a good feature to consider - I know there hasn't been much traction, but I'm going to tag in a number of feature areas that might want to take ownership here.

coredump17 commented 4 months ago

Any progress on this? Dashboards already supports Prometheus as a data source.

AdaptiveStep commented 3 months ago

Hi,

OpenSearch Community projects page lists Prometheus exporter plugin which can be installed very easily (like any other OpenSearch plugin). On top of that the Prometheus exporter plugin repo has a branch mixin which contains Grafana dashboard and Prometheus Alerts (if you do not want to build the dashboard from sources you can download its release here: https://github.com/aiven/prometheus-exporter-plugin-for-opensearch/releases/tag/mixin-1.0.0-rc.1).

Is that what you are looking for?

Regards, Lukáš

Why is this not a default plugin inside opensearch?!!! Its extremely useful!

lukas-vlcek commented 3 months ago

@AdaptiveStep There is a more general RFC to introduce metric framework directly into OpenSearch which should be able to support Prometheus servers as well.

dblock commented 3 months ago

Why is this not a default plugin inside opensearch?!!! Its extremely useful!

I think the maintainers of https://github.com/aiven/prometheus-exporter-plugin-for-opensearch should first voice whether they want their plugin to be included in the default distribution or not. Consider opening an issue there?

In general, every plugin we add proportionally increases the work to make a release because OpenSearch is tightly coupled, which isn't great.

saeed-mcu commented 3 months ago

@dblock prometheus-exporter-plugin-for-opensearch is very very essential and useful for performance analysis of opensearch and I think it MUST be added as default plugin inside opensearch as soon as possible

lukas-vlcek commented 3 months ago

I think it would make sense to bring the Prometheus exporter plugin a lot closer to the OpenSearch core and I would support this transition.

Let's discuss some cons and pros (feel free to add more, the following are just the ones I can think of right now).

Cons

Increased cost for the core maintainers and committers:

The main "downside" I can see is that this will add "yet another core plugin" that needs maintenance.
- However; I see this as temporary only because once the [RFC] Metrics Framework matures enough then the Prometheus exporter plugin should not be needed at all – see below in Pros section for further details.
- As long as I am the main maintainer of the Prometheus exporter plugin I do not think it is that important which repository/organization I do this work in – except for releases, which I do not have any privs to do if the code is in one of the OpenSearch upstream repositories.
New plugin PRs will probably need reviews from more people than a single one (currently it is me). Actually, I don't think this is a bad thing, but yes, technically this needs to be accounted for in the cost.

Pros

The [RFC] Metrics Framework should replace the need for Prometheus exporter plugin in a longer run (or at least that is my understanding). Having the Prometheus exporter plugin directly under the same "umbrella" the Metric Framework effort could benefit from having a real goal to aim for. I might be mistaken but it seems to me that the Metric Framework focuses a lot on Node Stats API but in practice useful metrics come also from other places (such as Node Info, Indices Stats or Cluster configuration to name a few and individual pieces of the data need to be aligned). This can be a way how to make sure the Metric Framework provides something that is really requested by existing users using Prometheus to monitor OpenSearch cluster(s).

Other things to consider

There is more to the monitoring story than just metric exporting from OpenSearch. In case of Prometheus/Grafana there are also Prometheus recording rules and alerts and Grafana dashboard. These parts can be currently found in the mixin branch of the Prometheus exporter plugin repo. They may still need a home... and I am not sure if upstream OpenSearch repo is the right place for it. On the other hand these parts are usually custom specific and ready-to-customize alternatives could be found elsewhere.

Regards, Lukáš

lukas-vlcek commented 2 months ago

opensearch-project / OpenSearch