[Campaign] Add distribution level Backward Compatiblity tests to all plugins

[zelinh]

Followup to this issue https://github.com/opensearch-build/issues/2870 when we onboarded BWC distribution tests to few plugins that have BWC tests running on plugin level already. However, there are numbers of plugins that are not included in our CI BWC tests on distribution bundle basis.

The reason we didn't onboard these plugins are:

• Plugins were new and recently added to our OpenSearch distribution so they don't have their own BWC tests yet.
• Plugins' BWC tests on plugins level were failing consistently and need fixed.

We are planning to add BWC tests to all plugins that we launch with the distribution bundle. This issue serves as a tracker to keep track of the BWC tests status of each plugins.

Component	Plugin level BWC tests	Distribution level BWC tests	Comment
alerting	✅	✅
anomaly-detection	✅	✅
asynchronous-search	✅	✅	https://github.com/opensearch-project/asynchronous-search/issues/517
reporting	✅	❌	https://github.com/opensearch-project/reporting/issues/946
index-management	✅	✅
k-NN	✅	❌	https://github.com/opensearch-project/k-NN/issues/1434
performance-analyzer	✅	❌	https://github.com/opensearch-project/performance-analyzer/issues/614
security	✅	❌	https://github.com/opensearch-project/security/issues/4005
sql	✅	✅
observability	✅	✅
notifications	✅	✅
cross-cluster-replication	✅	❌	https://github.com/opensearch-project/cross-cluster-replication/issues/1314
geospatial	❌	❌	https://github.com/opensearch-project/geospatial/issues/5
ml-commons	✅	❌	https://github.com/opensearch-project/ml-commons/issues/1962
security-analytics	❌	❌	https://github.com/opensearch-project/security-analytics/issues/470
neural-search	✅	✅	https://github.com/opensearch-project/neural-search/issues/202
custom-codecs	❌	❌	https://github.com/opensearch-project/custom-codecs/issues/167
flow-framework	❌	❌	https://github.com/opensearch-project/flow-framework/issues/787
skills	❌	❌	https://github.com/opensearch-project/skills/issues/354

[gaiksaya]

Hi @zelinh @bbarani Do we need to create child issues in all repositories missing the distribution level BWC tests?

[bbarani]

@zelinh Did we not create child issues in corresponding repos?

[rishabh6788]

@zelinh to create child issues and tag here.

[zelinh]

Child issues for requests on BWC distribution level tests are created in each component repo which doesn't have it right now. This issue will serve as an campaign to keep track of each of those issues.

[jmazanec15]

@zelinh is this change applicable to both 2.x and main? In AD, I see it was only added to 2.x branch: https://github.com/opensearch-project/anomaly-detection/pull/766.

[zelinh]

@zelinh is this change applicable to both 2.x and main? In AD, I see it was only added to 2.x branch: opensearch-project/anomaly-detection#766.

Since main branch is a major version change of OpenSearch. We expect there might/might not be breaking changes that fails the test and main branch is still under development. That's why we didn't add distribution level BWC test to main yet. If you find out that it's running on main branch. I'm good to add it now.