search

pjhampton / kibana-prometheus-exporter

Prometheus metrics for Kibana
Apache License 2.0
114 stars 37 forks source link

Kibana 7.10.2 - Backport bugfix #235

Closed rursprung closed 3 years ago

rursprung commented 3 years ago

as you might've heard elastic has changed the license of Elasticsearch for >= 7.11, there's no longer an Apache 2.0 licensed version available. thus we (and others) are currently on 7.10.2 and will eventually move to OpenSearch.

i've seen that you have made various bugfixes for newer versions. namely, #233 affects us as well and we'd need this bugfix for 7.10.2.

your versioning schema does not seem to allow for additional plugin releases for existing kibana releases since you 1:1 re-use the same version number from kibana (compared to e.g. vvanholl/elasticsearch-prometheus-exporter which has an additional sub-version-number) and you don't seem to have a release branch for 7.10.2 (as this was released off of your main branch; you do have a release/7.10.0 branch though which doesn't contain any newer 7.10 commits).

due to the license situation we're probably not the only consumers which will stay on 7.10.2 for the near future (and then move to OpenSearch). would it be possible to create a branch for 7.10.2, cherry-pick the fixes and then create a new release?

on a related note (but that should probably be a dedicated ticket): any interest from your side to support OpenSearch with your plugin?

pjhampton commented 3 years ago

as you might've heard elastic has changed the license of Elasticsearch for >= 7.11, there's no longer an Apache 2.0 licensed version available. thus we (and others) are currently on 7.10.2 and will eventually move to OpenSearch.

Thanks, @rursprung! Full disclosure, I work for Elastic and I wrote this plugin before I joined them as an engineer. Although I don't work on the Stack team, you will see me committing regularly to Kibana and occasionally to other components such as Elasticsearch. I am privy to the breaks / license changes etc.

i've seen that you have made various bugfixes for newer versions. namely, #233 affects us as well and we'd need this bugfix for 7.10.2.

That's fine. I do special releases when people need them. I am committed to supporting anything over 7.10. This plugin is used at a major investment bank exporting metrics for a huge fleet of kibana installs so I like to keep it operational for them. I will cut you a release with that fix this weekend.

your versioning schema does not seem to allow for additional plugin releases for existing kibana releases since you 1:1 re-use the same version number from kibana (compared to e.g. vvanholl/elasticsearch-prometheus-exporter which has an additional sub-version-number) and you don't seem to have a release branch for 7.10.2 (as this was released off of your main branch; you do have a release/7.10.0 branch though which doesn't contain any newer 7.10 commits).

due to the license situation we're probably not the only consumers which will stay on 7.10.2 for the near future (and then move to OpenSearch). would it be possible to create a branch for 7.10.2, cherry-pick the fixes and then create a new release?

I'm happy to adopt @vvanholl's versioning scheme going forward if people appreciate the consistency.

on a related note (but that should probably be a dedicated ticket): any interest from your side to support OpenSearch with your plugin?

If OpenSearch can keep its internal stats APIs consistent, I don't see why not. FWIW, you are the first person to ask me about this. @vvanholl will you be supporting OpenSearch in the native ES exporter going forward?

Edit: Apologies, pinged the wrong GH user.

rursprung commented 3 years ago

That's fine. I do special releases when people need them. I am committed to supporting anything over 7.10. This plugin is used at a major investment bank exporting metrics for a huge fleet of kibana installs so I like to keep it operational for them. I will cut you a release with that fix this weekend.

thanks a lot! please don't work on the weekend just for us!

I'm happy to adopt @vvanholl's versioning scheme going forward if people appreciate the consistency.

while it's not exactly following semver (as it has one digit too much) i think it is quite helpful for this use-case. or you can use the versioning schema used by Search Guard?

on a related note (but that should probably be a dedicated ticket): any interest from your side to support OpenSearch with your plugin?

If OpenSearch can keep its internal stats APIs consistent, I don't see why not. FWIW, you are the first person to ask me about this. @vvanholl will you be supporting OpenSearch in the native ES exporter going forward?

@aparo has created a fork for OpenSearch: aparo/opensearch-prometheus-exporter. at least for OpenSearch 1.x the APIs will stay compatible with Elasticsearch 7.10.2 (OpenSearch follows semver, so no breaking changes within a major release). so in the short-term your plugin would also work against OpenSearch Dashboards (though i'm not familiar with Kibana plugin development; for Elasticsearch/OpenSearch the plugin has to be duplicated due to java dependencies). i've also asked about potential a OpenSearch Dashboards prometheus exporter in the OpenSearch forums

pjhampton commented 3 years ago

Thanks for the nudge @rursprung 🎖️ I meditated on this for a while. I think creating a separate release for now suits as Elasticsearch + OpenSearch and going to diverge more and more as time goes on. Here you go https://github.com/pjhampton/kibana-prometheus-exporter/releases/tag/7.10.2-opensearch 👍

rursprung commented 3 years ago

thanks for the release, @pjhampton!

i'm a bit confused by the tag 7.10.2-opensearch: is this now for elastic kibana 7.10.2 or for opensearch dashboards 1.0.0 or somehow for both?

faust64 commented 3 years ago

I think it's meant for opensearch-dashboards-1.0.X. While the kibana version in index file is set to 7.14.1.

Have you tried to install it? There may be additional issues (#236), I didn't have time to dig further yet, ... hopefully soon.

pjhampton commented 3 years ago

Thanks @faust64 / @rursprung

There seem to be a couple of complexities I missed. I will report back on my return from vacation.

pjhampton commented 3 years ago

Thanks, guys. After some toying around I have decided not to support OpenSearch Dashboards through this project for the following reasons:

My recommendations are:

Sorry if anyone is disappointed. Thanks for starting this worthwhile convo @rursprung! ❤️

rursprung commented 3 years ago

no worries. it actually wasn't my goal to ask for opensearch support with this ticket - but thanks for looking into this! instead, we for now still have elasticsearch 7.10.2 and would like to use your plugin there, which is why we'd need the backports of these bugfixes for 7.10.2.

pjhampton commented 3 years ago

Of course, @rursprung - here is the fix: https://github.com/pjhampton/kibana-prometheus-exporter/releases/tag/7.10.2

rursprung commented 3 years ago

Of course, @rursprung - here is the fix: https://github.com/pjhampton/kibana-prometheus-exporter/releases/tag/7.10.2

i tested the v2 attachment there but it fails to start on a fresh kibana 7.10.2 installation:

Manifest for plugin "kibanaPrometheusExporter" contains the following unrecognized properties: owner. (invalid-manifest, C:\[..]\kibana-7.10.2-windows-x86_64\plugins\kibanaPrometheusExporter\kibana.json)

note: running kibana-plugin install worked fine, the failure happens when trying to start kibana itself.

faust64 commented 3 years ago

Seee #236, I've seen similar errors, you may need to patch the kibana.json file