BootStrapTests#testTriggeredWatchLoading can fail assertion

jtibshirani commented 3 years ago

Build scan: https://gradle-enterprise.elastic.co/s/nnardp2pu6tau

Repro line:

./gradlew ':x-pack:plugin:watcher:internalClusterTest' \
  --tests "org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.testTriggeredWatchLoading" \
  -Dtests.seed=7DB0FF08E8B2DB2C -Dtests.security.manager=true -Dtests.locale=it-CH \
  -Dtests.timezone=Europe/Samara -Druntime.java=11

Reproduces locally?: no

Applicable branches: master (at least)

Failure history: fails sporadically but regularly, around twice a month

Failure excerpt:

 2> java.lang.AssertionError: Count is 8 hits but 4 was expected.  Total shards: 1 Successful shards: 1 & 0 shard failures:
        at __randomizedtesting.SeedInfo.seed([7DB0FF08E8B2DB2C:445E9FB2D758E3D3]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount(ElasticsearchAssertions.java:270)
        at org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.lambda$assertSingleExecutionAndCompleteWatchHistory$3(BootStrapTests.java:288)
        at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:955)
        at org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.assertSingleExecutionAndCompleteWatchHistory(BootStrapTests.java:274)
        at org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.testTriggeredWatchLoading(BootStrapTests.java:269)

elasticmachine commented 3 years ago

Pinging @elastic/es-core-features (Team:Core/Features)

pugnascotia commented 3 years ago

Still failing - https://gradle-enterprise.elastic.co/s/bcxhauxgbm6rk

mark-vieira commented 3 years ago

This looks to fail about once a day. Just enough to be annoying.

jrodewig commented 3 years ago

Another failure: https://gradle-enterprise.elastic.co/s/spt6iohiknquw

vamuzumd commented 3 years ago

Another failure for me -> https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request-2/2846/testReport/junit/org.elasticsearch.xpack.watcher.test.integration/BootStrapTests/testTriggeredWatchLoading/

astefan commented 3 years ago

A more recent failure: https://gradle-enterprise.elastic.co/s/hspurs4o6yl3g Failed twice in the last two weeks.

org.elasticsearch.xpack.watcher.test.integration.BootStrapTests > testTriggeredWatchLoading FAILED
    java.lang.AssertionError: Count is 10 hits but 5 was expected.  Total shards: 1 Successful shards: 1 & 0 shard failures:
        at __randomizedtesting.SeedInfo.seed([F4277DDFA8895231:CDC91D6597636ACE]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount(ElasticsearchAssertions.java:258)
        at org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.lambda$assertSingleExecutionAndCompleteWatchHistory$3(BootStrapTests.java:289)
        at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:977)
        at org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.assertSingleExecutionAndCompleteWatchHistory(BootStrapTests.java:275)
        at org.elasticsearch.xpack.watcher.test.integration.BootStrapTests.testTriggeredWatchLoading(BootStrapTests.java:270)

original-brownbear commented 2 years ago

Failed again here: https://gradle-enterprise.elastic.co/s/o2zscsukx2366/tests/:x-pack:plugin:watcher:internalClusterTest/org.elasticsearch.xpack.watcher.test.integration.BootStrapTests/testTriggeredWatchLoading

jtibshirani commented 2 years ago

I think the same failure popped up again. I forgot we had this issue and accidentally opened a new one: https://github.com/elastic/elasticsearch/issues/85912.

elastic / elasticsearch

BootStrapTests#testTriggeredWatchLoading can fail assertion #67729