Make unit tests stable

[mocobeta]

Some of unit tests are still unstable.

I posted a question about usage of LuceneTestCase to lucene user's mailing list. Here is the thread archive: http://mail-archives.apache.org/mod_mbox/lucene-java-user/201808.mbox/browser

[mocobeta]

Here are random seeds (i.e. test parameter combinations) those produces assertion errors:

mvn test -Dtest=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=CA70FBB57042AF34

NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=22543, maxDocsPerChunk=448, blockSize=3), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=22543, blockSize=3)), sim=RandomSimilarity(queryNorm=true): {}, locale=el-CY, timezone=Pacific/Noumea

mvn test -Dtest=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=B00CDBB7AA2F2EA9

NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=1, maxDocsPerChunk=243, blockSize=229), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=1, blockSize=229)), sim=RandomSimilarity(queryNorm=true): {}, locale=ar-YE, timezone=America/Montreal

mvn test -Dtest=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=6F3AD2CDDC481719

NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=14493, maxDocsPerChunk=9, blockSize=10), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=14493, blockSize=10)), sim=RandomSimilarity(queryNorm=true): {}, locale=ar-LB, timezone=Asia/Aqtau

mvn test -Dtest=CommitsImplTest -Dtests.method=testGetCommit_generation_notfound -Dtests.seed=CCFC7423CFDCC16A

NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=sr-RS, timezone=America/Santiago

mvn test -Dtest=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=C2F6CB0A0E641E29

NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=3, maxDocsPerChunk=675, blockSize=817), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=3, blockSize=817)), sim=RandomSimilarity(queryNorm=true): {}, locale=nl, timezone=Asia/Manila

mvn test -Dtest=CommitsImplTest -Dtests.method=testListCommits -Dtests.seed=12EB3E3B19E3AC89

NOTE: test params are: codec=Asserting(Lucene70): {f1=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=359, maxMBSortInHeap=5.195673810400375, sim=RandomSimilarity(queryNorm=false): {}, locale=tr, timezone=Indian/Cocos

mvn test -Dtest=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=A6ABF40276C979CF

NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=16938, maxDocsPerChunk=320, blockSize=258), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=16938, blockSize=258)), sim=RandomSimilarity(queryNorm=false): {}, locale=es-PY, timezone=Asia/Jakarta

mvn test -Dtestcase=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=FB9BFAF87DBD805D

NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=6289, maxDocsPerChunk=1, blockSize=576), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=6289, blockSize=576)), sim=RandomSimilarity(queryNorm=true): {}, locale=de, timezone=Asia/Jerusalem

mvn test -Dtestcase=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=4D0A3252A6F3F2F3

NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=16755, maxDocsPerChunk=249, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=16755, blockSize=1)), sim=RandomSimilarity(queryNorm=false): {}, locale=id-ID, timezone=Europe/Ljubljana

mvn test -Dtestcase=CommitsImplTest -Dtests.method=testGetSegmentAttributes -Dtests.seed=90B0C1FA836B2017

NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=1, maxDocsPerChunk=499, blockSize=9), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=1, blockSize=9)), sim=RandomSimilarity(queryNorm=true): {}, locale=en-IE, timezone=Canada/Yukon

[mocobeta]

@msokolov I think you are an expert in this area :) , so I just would like to share you the results above. We'd appreciate if you share us your intuitions / suggestions. I am not rushing to resolve it, so please do not mind if you don't want to respond.

[mocobeta]

I got the cause of testGetSegmentAttributes() failures: Subclasses of CompressingCodec do not store any segment attributes. So can we ignore all subclasses of CompressingCodec for our tests (by @LuceneTestCase.SuppressCodecs annotation ) ? Those are contained in test-framework, but there are no mentions about segment attributes. http://lucene.apache.org/core/7_5_0/test-framework/org/apache/lucene/codecs/compressing/CompressingCodec.html

[msokolov]

Nice work!