asfimport
commented
5 years ago Chris M. Hostetter (@hossman) (migrated from JIRA)
I'm attaching a patch with my test – there are some nocommits related to ways i think the test could/should be improved, but as it stands it doesn't take much beasting to find some seeds that reproduce fairly reliably for me (on master)...
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterExceptions -Dtests.method=testRandomCorruptionIsTragic -Dtests.seed=48825E06EC3D16C1 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=is-IS -Dtests.timezone=Etc/Zulu -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.47s | TestIndexWriterExceptions.testRandomCorruptionIsTragic <<<
[junit4] > Throwable #1: java.lang.AssertionError: index update encountered throwable, but no tragic event recorded: java.nio.file.NoSuchFileException: _0_LuceneFixedGap_0.tib
[junit4] > at __randomizedtesting.SeedInfo.seed([48825E06EC3D16C1:A7D1C7299F88ED1B]:0)
[junit4] > at org.apache.lucene.index.TestIndexWriterExceptions.testRandomCorruptionIsTragic(TestIndexWriterExceptions.java:2146)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {field=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, maxPointsInLeafNode=1182, maxMBSortInHeap=6.385378646488135, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@1139fb33), locale=is-IS, timezone=Etc/Zulu
[junit4] 2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=194368568,total=249561088
[junit4] 2> NOTE: All tests run in this JVM: [TestIndexWriterExceptions]
[junit4] Completed [1/1 (1!)] in 0.91s, 1 test, 1 failure <<< FAILURES! [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterExceptions -Dtests.method=testRandomCorruptionIsTragic -Dtests.seed=662E8085B38FC223 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-NI -Dtests.timezone=Asia/Jerusalem -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.73s | TestIndexWriterExceptions.testRandomCorruptionIsTragic <<<
[junit4] > Throwable #1: java.lang.AssertionError: index update encountered throwable, but no tragic event recorded: java.nio.file.NoSuchFileException: _0_TestBloomFilteredLucenePostings_0.doc
[junit4] > at __randomizedtesting.SeedInfo.seed([662E8085B38FC223:897D19AAC03A39F9]:0)
[junit4] > at org.apache.lucene.index.TestIndexWriterExceptions.testRandomCorruptionIsTragic(TestIndexWriterExceptions.java:2146)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {field=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{}, maxPointsInLeafNode=191, maxMBSortInHeap=6.402103555115238, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@44d3ba93), locale=es-NI, timezone=Asia/Jerusalem
[junit4] 2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=229171096,total=249561088
[junit4] 2> NOTE: All tests run in this JVM: [TestIndexWriterExceptions]
[junit4] Completed [1/1 (1!)] in 1.18s, 1 test, 1 failure <<< FAILURES!It's not clear to me if this is truly a bug in the IndexWriter's exception handling when propogating these NoSuchFileException or if MockDirectoryWrapper is being "too evil" and creating situations we don't expect IndexWriter to be able to recognize as tragic – allthough in my opinion it certainly seems like IW should treat files being deleted out from under it as tragic.
Backstory...
Solr has a "LeaderTragicEventTest" which uses MockDirectoryWrapper's
corruptFilesto introduce corruption into the "leader" node's index and then assert that this solr node gives up it's leadership of the shard and another replica takes over.This can currently fail sporadically (but usually reproducibly - see SOLR-13237) due to the leader not giving up it's leadership even after the corruption causes an update/commit to fail. Solr's leadership code makes this decision after encountering an exception from the IndexWriter based on wether
IndexWriter.getTragicException()is (non-)null.While investigating this, I created an isolated Lucene-Core equivilent test that demonstrates the same basic situation:
getTragicException()is now non-null.It's fairly easy to make my new test fail reproducibly – in every situation I've seen the underlying exception is a
NoSuchFileException(ie: the randomly introduced corruption was to delete some file).Migrated from LUCENE-8692 by Chris M. Hostetter (@hossman), updated Mar 12 2019 Attachments: LUCENE-8692_test.patch, LUCENE-8692.patch (versions: 3) Linked issues: