Open valeriy42 opened 2 years ago
Pinging @elastic/es-data-management (Team:Data Management)
It looks like this one is a timeout from the stack trace above, and from this in the logs:
1> [2022-03-07T10:22:07,024][INFO ][o.e.x.r.FullClusterRestartIT] [testWatcherWithApiKey] before test
1> [2022-03-07T10:22:07,027][INFO ][o.e.x.r.FullClusterRestartIT] [testWatcherWithApiKey] initializing REST clients against [http://[::1]:33333, http://127.0.0.1:45153, http://[::1]:45317, http://127.0.0.1:34561]
1> [2022-03-07T10:22:41,230][INFO ][o.e.x.r.FullClusterRestartIT] [testWatcherWithApiKey] after test
Running it locally, it only takes 4-5 seconds. So I'm not sure why it's so much slower sometimes on the CI server.
Another failure: https://gradle-enterprise.elastic.co/s/fa5zfc5on4v5w
Still failing: https://gradle-enterprise.elastic.co/s/mfofju4qvgkqg
One more failure: https://gradle-enterprise.elastic.co/s/zj3onfpgzyjyi
And one more: https://gradle-enterprise.elastic.co/s/t4f5rxsye47qy
A similar failure but with
./gradlew ':x-pack:qa:full-cluster-restart:v7.3.2#oldClusterTest' -Dtests.class="org.elasticsearch.xpack.restart.FullClusterRestartIT" -Dtests.method="testWatcherWithApiKey" -Dtests.seed=D916BB57EB898DDF -Dtests.bwc=true -Dtests.locale=es-PE -Dtests.timezone=Asia/Vientiane -Druntime.java=17
Another one at https://gradle-enterprise.elastic.co/s/xqrve2db4nxiq/console-log?task=:x-pack:qa:full-cluster-restart:v7.17.10%23bwcTest
:
:x-pack:qa:full-cluster-restart:v7.17.10#bwcTest FAILED
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF8
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF8
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF8
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF8
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF8
org.elasticsearch.xpack.restart.FullClusterRestartIT > testWatcherWithApiKey {cluster=UPGRADED} FAILED
java.lang.AssertionError: version increased: [true], executed: [false]
Expected: is <true>
but: was <false>
at __randomizedtesting.SeedInfo.seed([D08E51FCB5D10E4C:39D59B8A4AE21F65]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.elasticsearch.xpack.restart.FullClusterRestartIT.lambda$testWatcherWithApiKey$1(FullClusterRestartIT.java:279)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1112)
at org.elasticsearch.xpack.restart.FullClusterRestartIT.testWatcherWithApiKey(FullClusterRestartIT.java:270)
Tests with failures:
- org.elasticsearch.xpack.restart.FullClusterRestartIT.testWatcherWithApiKey {cluster=UPGRADED}
78 tests completed, 1 failed, 14 skipped
Slightly different error to the headline above
java.lang.AssertionError: version increased: [true], executed: [false]
Expected: is <true>
but: was <false>
at org.elasticsearch.xpack.restart.FullClusterRestartIT.lambda$testWatcherWithApiKey$1(FullClusterRestartIT.java:279)
at org.elasticsearch.xpack.restart.FullClusterRestartIT.testWatcherWithApiKey(FullClusterRestartIT.java:270)
Maybe we're running into https://github.com/elastic/elasticsearch/issues/69842? I can't find any evidence of it, but it does seem that watcher is just not running for some reason.
Seems to be failing regularly with the same error recently.
Build scan: https://gradle-enterprise.elastic.co/s/b3aibsuqwx4ww/tests/:x-pack:qa:full-cluster-restart:v7.2.0%23oldClusterTest/org.elasticsearch.xpack.restart.FullClusterRestartIT/testWatcherWithApiKey
Reproduction line:
./gradlew ':x-pack:qa:full-cluster-restart:v7.2.0#oldClusterTest' -Dtests.class="org.elasticsearch.xpack.restart.FullClusterRestartIT" -Dtests.method="testWatcherWithApiKey" -Dtests.seed=89F33062A7870C11 -Dtests.bwc=true -Dtests.locale=ar-KW -Dtests.timezone=Europe/Brussels -Druntime.java=17
Applicable branches: 8.0
Reproduces locally?: Didn't try
Failure history: https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.xpack.restart.FullClusterRestartIT&tests.test=testWatcherWithApiKey
Failure excerpt: