search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.81k stars 1.82k forks source link

[AUTOCUT] Gradle Check Flaky Test Report for RemoteSplitIndexIT #14296

Open opensearch-ci-bot opened 5 months ago

opensearch-ci-bot commented 5 months ago

Flaky Test Report for RemoteSplitIndexIT

Noticed the RemoteSplitIndexIT has some flaky, failing tests that failed during post-merge actions.

Details

Git Reference Merged Pull Request Build Details Test Name
0cd002dacc94459bb5a7b4642bfc33d2aa90ad83 15055 43778 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
1386a9b902c4af0e3cb88a6e7e16861970415b76 13930 39885 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
33876069a24bbf1ece4dd5cde8b5b3df72ff0d10 14954 43371 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
392e666e0e13dfc16923e8476e2c42f19a82c818 14487 41458 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testSplitIndexPrimaryTerm
3b0ea6525af8a04c60288ba133049b7aa954b29e 14863 42979 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
563375de28b16870ab42b9fb4260127598d47d91 14187 41622 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
5ad0f5dc1303bf63c973cd93987077d9748ab167 14203 40762 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
5b1945419dc8da8b1ce1ce46cb1f163e61b08018 13801 39904 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
784f7d3264eb7efdb3c4597d2e72650ddbd5d39f 14214 40795 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
887698d22fbca28f29c8ffc0f635228ac209d6a1 15132 44111 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
8b1021518345d9b5d36a08302dc8871eeae6fd2b 14869 43078 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
8f95735f0b75eb3fca117f82ebeda5fd32c55f6b 15172 44301 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
913013bd5c6b43d8337a97a7753bc2f10f36eae4 13948 39666 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
a211f1d5986e3fe6c807ae3b26b0157de621a55f 14820 42761 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
a6c86e7774df984f45b506830c9bf581746e92de 13906 39544 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
af5a3d1955e6cc27bd1e97f3e9861167a0edad1f 16147 48706 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
b9ca5a8e24673ed38cab736ffbd57479de241553 14027 40019 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
bc39354db3923b2aedd13c15d92f51d2038b3489 14124 40730 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
c1e138fb8ca8f63834196aed4f0a32b195ca5a1a 15376 45125 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
c38dfef55610aa5a1f713b22e54e2f282f327ef0 14195 40750 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
d2757f74e50b66b7cda494f6d06936d6d32bb2c1 14250 40921 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
d4bdc7a601a8e212ddffcdbcd95bba4fff78a872 15029 44176 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
d56d8c88e07ae416d41197b05103ea2dba393967 14489 41572 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
ee17ecacc0daeed27e4b9640a0ad0e5c3f766960 14630 45612 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
f2fd8047e1669395742657760ea35c11f50368e0 14166 40663 org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the RemoteSplitIndexIT class are:

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

andrross commented 5 months ago

The "classMethod" failure appears to be due to a file leak:

RemoteSplitIndexIT > classMethod FAILED
    java.lang.RuntimeException: file handle leaks: [InputStream(/var/jenkins/workspace/gradle-check/search/server/build/testrun/internalClusterTest/temp/org.opensearch.action.admin.indices.create.RemoteSplitIndexIT_3726536651049A1A-001/tempDir-002/repos/cjWLgWIQEY/T11011111101111/6z6VohLmTKe07eW5Z98zQA/1/translog/data/2/translog-11.tlog)]
        at __randomizedtesting.SeedInfo.seed([3726536651049A1A]:0)
        at org.apache.lucene.tests.mockfile.LeakFS.onClose(LeakFS.java:63)
        at org.apache.lucene.tests.mockfile.FilterFileSystem.close(FilterFileSystem.java:69)
        at org.apache.lucene.tests.mockfile.FilterFileSystem.close(FilterFileSystem.java:70)
        at org.apache.lucene.tests.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:223)
        at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.****/java.lang.Thread.run(Thread.java:1583)
./gradlew ':server:internalClusterTest' --tests "org.opensearch.action.admin.indices.create.RemoteSplitIndexIT" -Dtests.seed=3726536651049A1A -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en -Dtests.timezone=Etc/UTC -Druntime.java=21
dblock commented 2 months ago

[Catch All Triage - 1, 2, 3, 4, 5]

prachi-gaonkar commented 1 month ago

Hi Team, we are also facing same issues with ppc64le

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex" -Dtests.seed=71FF58527F2E975B -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=fr-CH -Dtests.timezone=Asia/Saigon -Druntime.java=21

RemoteSplitIndexIT > testCreateSplitIndex FAILED UncategorizedExecutionException[Failed execution]; nested: IOException[Failed to upload 2 files during transfer]; at __randomizedtesting.SeedInfo.seed([71FF58527F2E975B:9BF8EE873AE5D59]:0) at app//org.opensearch.action.support.AdapterActionFuture.unwrapEsException(AdapterActionFuture.java:102) at app//org.opensearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:57) at app//org.opensearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:73) at app//org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex(RemoteSplitIndexIT.java:414)

    Caused by:
    java.io.IOException: Failed to upload 2 files during transfer
        at org.opensearch.index.translog.transfer.TranslogTransferManager.transferSnapshot(TranslogTransferManager.java:199)
        at org.opensearch.index.translog.RemoteFsTranslog.upload(RemoteFsTranslog.java:426)
        at org.opensearch.index.translog.RemoteFsTranslog.prepareAndUpload(RemoteFsTranslog.java:409)
        at org.opensearch.index.translog.RemoteFsTranslog.ensureSynced(RemoteFsTranslog.java:341)
        at org.opensearch.index.translog.Translog.ensureSynced(Translog.java:837)
        at org.opensearch.index.translog.InternalTranslogManager.ensureTranslogSynced(InternalTranslogManager.java:184)
        at org.opensearch.index.engine.InternalEngine.ensureTranslogSynced(InternalEngine.java:605)
        at org.opensearch.index.shard.IndexShard.lambda$createTranslogSyncProcessor$44(IndexShard.java:4441)
        at org.opensearch.index.shard.IndexShard$6.write(IndexShard.java:4455)
prachi-gaonkar commented 4 weeks ago

Hi Team is there any update on this issue?