search

opensearch-project / index-management

🗃 Automate periodic data operations, such as deleting indices at a certain age or performing a rollover at a certain size
https://opensearch.org/docs/latest/im-plugin/index/
Apache License 2.0
52 stars 107 forks source link

[BUG] `IndexStateManagementSecurityBehaviorIT` test fails when run twice against a remote cluster #1101

Open DarshitChanpura opened 4 months ago

DarshitChanpura commented 4 months ago

What is the bug? While debugging the fix for https://github.com/opensearch-project/index-management/pull/1100, I ran into:

REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT.test add policy" -Dtests.seed=5DCE79AE35A726F7 -Dtests.security.manager=false -Dtests.locale=de-CH -Dtests.timezone=Africa/Djibouti -Druntime.java=17

org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT > test add policy FAILED
    java.lang.AssertionError: expected:<201> but was:<200>
        at __randomizedtesting.SeedInfo.seed([5DCE79AE35A726F7:6DEFFE55BAD2DB76]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:120)
        at org.junit.Assert.assertEquals(Assert.java:146)
        at org.opensearch.indexmanagement.SecurityRestTestCase.executeRequest(SecurityRestTestCase.kt:381)
        at org.opensearch.indexmanagement.SecurityRestTestCase.createUser(SecurityRestTestCase.kt:398)
        at org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT.setupUsersAndRoles(IndexStateManagementSecurityBehaviorIT.kt:82)

Upon further investigation, it was discovered that IndexStateManagementSecurityBehaviorIT is failing because of https://github.com/opensearch-project/index-management/blob/main/src/test/kotlin/org/opensearch/indexmanagement/SecurityRestTestCase.kt#L399 check inside SecurityRestTestCase.kt. If internal user doesn’t exist in the system then 201 CREATED is returned which is expected by the test. But, if internal user already exists then 200 OK is returned which is unexpected.

This issue only occurs when tests are run more than once against an existing cluster. If the cluster is tore-down and re-started this issue will not occur. This is because the data is wiped with teardown.

How can one reproduce the bug? Steps to reproduce the behavior:

  1. Start a full distribution opensearch cluster
  2. Run this twice: ./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch-cluster" -Dhttps=true -Duser=admin -Dpassword=<your-admin-password>
  3. See error

What is the expected behavior? Maybe this isn't a bug as the test are not expected to be run against remote cluster twice without tearing the cluster down. But if it is expected to be run more than once, then the test should not fail.

DarshitChanpura commented 4 months ago

When running it twice with this command:

./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch-cluster" -Dhttps=true -Duser=admin -Dpassword=<your-admin-password> --tests "org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT.test add policy"
I couldn't reproduce it again: ```zsh ➜ index-management git:(2.12) ✗ ./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch-cluster" -Dhttps=true -Duser=admin -Dpassword=myStrongPassword123! --tests "org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT.test add policy" ======================================= OpenSearch Build Hamster says Hello! Gradle Version : 8.5 OS Info : Linux 5.10.205-172.807.amzn2int.x86_64 (amd64) JDK Version : 17 (Amazon Corretto JRE) JAVA_HOME : /usr/lib/jvm/java-17-amazon-corretto.x86_64 Random Testing Seed : F18D86075F680416 In FIPS 140 mode : false ======================================= Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.5/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD SUCCESSFUL in 34s 13 actionable tasks: 2 executed, 11 up-to-date ➜ index-management git:(2.12) ✗ ./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch-cluster" -Dhttps=true -Duser=admin -Dpassword=myStrongPassword123! --tests "org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT.test add policy" ======================================= OpenSearch Build Hamster says Hello! Gradle Version : 8.5 OS Info : Linux 5.10.205-172.807.amzn2int.x86_64 (amd64) JDK Version : 17 (Amazon Corretto JRE) JAVA_HOME : /usr/lib/jvm/java-17-amazon-corretto.x86_64 Random Testing Seed : A18E873E4F7FCED1 In FIPS 140 mode : false ======================================= Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.5/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD SUCCESSFUL in 33s 13 actionable tasks: 2 executed, 11 up-to-date ➜ index-management git:(2.12) ✗ ./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch-cluster" -Dhttps=true -Duser=admin -Dpassword=myStrongPassword123! --tests "org.opensearch.indexmanagement.IndexStateManagementSecurityBehaviorIT.test add policy" ======================================= OpenSearch Build Hamster says Hello! Gradle Version : 8.5 OS Info : Linux 5.10.205-172.807.amzn2int.x86_64 (amd64) JDK Version : 17 (Amazon Corretto JRE) JAVA_HOME : /usr/lib/jvm/java-17-amazon-corretto.x86_64 Random Testing Seed : 355E72463C6067AB In FIPS 140 mode : false ======================================= Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.5/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD SUCCESSFUL in 34s 13 actionable tasks: 2 executed, 11 up-to-date ➜ index-management git:(2.12) ✗ ```

Maybe this is not a bug?

dblock commented 2 weeks ago

Catch All Triage - 1 2 3 4 5