search

opensearch-project / cross-cluster-replication

Synchronize your data across multiple clusters for lower latencies and higher availability
https://opensearch.org/docs/latest/replication-plugin/index/
Apache License 2.0
47 stars 57 forks source link

[ACTION NEEDED] Fix flaky integration tests at distribution level #1362

Closed gaiksaya closed 2 months ago

gaiksaya commented 4 months ago

What is the bug? It was observed in 2.13.0 and previous other releases that this component manually signed off on the release for failing integration tests. See https://github.com/opensearch-project/opensearch-build/issues/4433#issuecomment-2026436873 The flakiness of the test runs take a lot of time from the release team to collect go/no-go decision and significantly lower the confidence in the release bundles.

How can one reproduce the bug? Steps to reproduce the behavior:

  1. Run integration testing for altering and see the failures.
  2. Issues can be reproduced using the steps declared in AUTOCUT issues for failed integration testing

What is the expected behavior? Tests should be consistently passing.

Do you have any additional context? Please note that this is a hard blocker for 2.14.0 release as per the discussion here

ankitkala commented 4 months ago

Related issue: https://github.com/opensearch-project/opensearch-build/issues/4610

bbarani commented 4 months ago

FYI... We will ignore the failing test for Deb / RPM in 2.14.0 release if we cannot come up with a solution before the release timeline. We should close this gap for 2.15.0 release though. CC: @rishabh6788 @gaiksaya @peterzhuamazon

gaiksaya commented 4 months ago

Adding 2.14.0 release manager @rishabh6788

dblock commented 2 months ago

In 2.15 we ran with automation end-to-end, closing.

Catch All Triage - 1 2 3 4 5 6