search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.45k stars 973 forks source link

TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader failure due to DWPT race condition #13352

Open benwtrent opened 1 month ago

benwtrent commented 1 month ago

Description

This replicates rarely for me locally. To me this indicates a race condition instead of some logic error.

    java.lang.AssertionError: seqNo=114262 vs maxSeqNo=114261
        at __randomizedtesting.SeedInfo.seed([905AD19F00667602]:0)
        at org.apache.lucene.index.DocumentsWriterDeleteQueue.getNextSequenceNumber(DocumentsWriterDeleteQueue.java:567)
        at org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:178)
        at org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:155)
        at org.apache.lucene.index.DocumentsWriterPerThread.finishDocuments(DocumentsWriterPerThread.java:328)
        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:273)
        at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:425)
        at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1558)
        at org.apache.lucene.index.IndexWriter.softUpdateDocuments(IndexWriter.java:1603)
        at org.apache.lucene.tests.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:315)
        at org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:781)
        at java.base/java.lang.Thread.run(Thread.java:1583)

This is very similar to: https://github.com/elastic/elasticsearch/issues/105122#issuecomment-2098412230

Either we are updating maxSeqNo incorrectly, or there is a race condition in DWPT that both tests have hit

Gradle command to reproduce

./gradlew test --tests TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader -Dtests.seed=905AD19F00667602 -Dtests.locale=chr-Cher-US -Dtests.timezone=MST7MDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8