search

opensearch-project / performance-analyzer

📈 Get detailed performance metrics from your cluster independently of the Java Virtual Machine (JVM)
https://opensearch.org/docs/latest/monitoring-plugins/pa/index/
Apache License 2.0
32 stars 66 forks source link

[FEATURE] Onboard distribution level BWC bundle test #614

Open zelinh opened 5 months ago

zelinh commented 5 months ago

Currently performance-analyzer repo has BWC test on plugin level and only tested within the repo. We have a framework for BWC distribution level test available and you could utilize it and enable distribution level BWC test.

With this framework, you would be able to spin up the test cluster with the latest distribution bundle that engineering effectiveness team generated that installs all the other components. This test will be more comprehensive as it will test the exact product we ship to the customers.

Onboarding this distribution level BWC test can also enable engineering effectiveness team to run BWC tests every time ee team generates a new bundle distribution across each of the plugin through CI

More onboarding info can be found here: https://github.com/opensearch-project/opensearch-plugins/blob/main/TESTING.md#bwc-tests-on-distribution-bundle-level

What solution would you like? Similar to what we did to the anomaly-detection plugin. https://github.com/opensearch-project/anomaly-detection/pull/766 Enable bundle level BWC test based on the BWC test you already have.

dblock commented 2 weeks ago

Catch All Triage - 1 2 3 4 5

dblock commented 2 weeks ago

@zelinh I assume this wasn't done and remains valid?

zelinh commented 2 weeks ago

This issue remains valid. We provided the platform for distribution level BWC tests and onboarded the tests to some of plugins. We are requesting each plugin to onboard to these bundle level BWC tests so we could run BWC tests to the bundle we generate and release.