opensearch-project / opensearch-api-specification

API specification for OpenSearch
Apache License 2.0
35 stars 62 forks source link

[BUG] Replication tests are flaky (index [books-names] blocked by: [FORBIDDEN/1000/index read-only(cross-cluster-replication)) #695

Closed dblock closed 3 days ago

dblock commented 3 days ago

What is the bug?

OpenSearch 2.18.0

ERROR   autofollow.yaml (tests/plugins/replication/autofollow.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to prologues or epilogues.
  /_plugins/_replication/_autofollow
  /_plugins/_replication/autofollow_stats

    ERROR   EPILOGUES
        PASSED  POST /_plugins/_replication/{index}/_stop
        ERROR   DELETE /books-names (index [books-names] blocked by: [FORBIDDEN/1000/index read-only(cross-cluster-replication)];)
        PASSED  DELETE /books

ERROR   index.yaml (tests/plugins/replication/index.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to prologues or epilogues.
  /_plugins/_replication/{index}/_start
  /_plugins/_replication/{index}/_update

    ERROR   CHAPTERS
        ERROR   Start replication.
            PASSED  PARAMETERS
                PASSED  index
            PASSED  REQUEST BODY
            ERROR   RESPONSE STATUS (Expected status 200, but received 400: application/json. Cant use same index again for replication. Delete the index:books-names)
            SKIPPED RESPONSE PAYLOAD BODY
            SKIPPED RESPONSE PAYLOAD SCHEMA
            SKIPPED RESPONSE OUTPUT VALUES
        SKIPPED Update settings.
    ERROR   EPILOGUES
        ERROR   POST /_plugins/_replication/{index}/_stop (No replication in progress for index:books-names)
        PASSED  DELETE /books-names
        PASSED  DELETE /books

PASSED  stats.yaml (tests/plugins/replication/stats.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to prologues or epilogues.
  /_plugins/_replication/{index}/_status
  /_plugins/_replication/leader_stats
  /_plugins/_replication/follower_stats
  /_plugins/_replication/autofollow_stats

Tested 7/5[28](https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694#step:7:29) paths.
Writing coverage/test-spec-coverage-2.18.0-218a289a83d99f103c6e063e594[29](https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694#step:7:30)0a9879b80c1b2c08[30](https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694#step:7:31)15fa9752b7a36c9db.json ...
Error: Process completed with exit code 1.

How can one reproduce the bug?

https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694

What is the expected behavior?

Tests to always pass.

What is your host/environment?

GHA.

Do you have any screenshots?

Screenshot 2024-11-25 at 11 46 40 AM

Do you have any additional context?

Maybe a simple retry can work around this or use books in one set of tests and movies in another?

dblock commented 3 days ago

@Tokesh maybe you want to take a look when you get a chance

Tokesh commented 3 days ago

@dblock Oh! I'm sorry about that. I've already prepared a PR for your review. You can check it now.