bongohrtech / lucenenet

Mirror of Apache Lucene.Net
Apache License 2.0
0 stars 0 forks source link

Known Failing Tests on Lucene.Net #127

Open bongohrtech opened 5 years ago

bongohrtech commented 5 years ago

Up for grabs. Please contribute to Lucene.Net.

All 8000+ tests pass most of the time, but there are still a few that fail under the random conditions we test them in. This is a complete list of the known tests that fail and what is known about them. Note that these may not be all of the ways the tests can fail.

  1. Lucene.Net.Search.TestSearchAfter::TestQueries()
    • Related to current culture
    • Happens on netcoreapp2.1 on macOS
    • Happens on netcoreapp1.0 on Linux
  2. Lucene.Net.Util.TestVersionComparer::TestVersions()
    • Related to current culture
    • Happens on netcoreapp2.1 on macOS
  3. Lucene.Net.Index.TestIndexWriter::TestTwoThreadsInterruptDeadlock()
    • Happens on every target framework/OS
  4. Lucene.Net.Index.TestIndexWriter::TestThreadInterruptDeadlock()
    • Happens on every target framework/OS
    • Fails with message:
      FAILED; unexpected exception
       System.InvalidOperationException: cannot close: prepareCommit was already called with no corresponding call to commit
       at Lucene.Net.Index.IndexWriter.CloseInternal(Boolean waitForMerges, Boolean doFlush) in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line 1157
       at Lucene.Net.Index.IndexWriter.Dispose(Boolean waitForMerges) in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line 1096
       at Lucene.Net.Index.IndexWriter.Dispose() in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line 1053
       at Lucene.Net.Index.TestIndexWriter.IndexerThreadInterrupt.Run() in F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestIndexWriter.cs:line 1223
    • The error message doesn't occur when using lock (this} in IndexWriter.CommitInternal() to make PrepareCommitInternal() and FinishCommit() atomic, but the test still fails and the extra lock causes deadlocks in other tests. For some reason, the existing commitLock isn't completely atomic in some (unknown) case.
    • The test does not fail if both of the following lines are commented in TestIndexWriter.IndexerThreadInterrupt::Run()
      w.UpdateDocument(new Term("id", idField.GetStringValue()), doc);
      w.AddDocument(doc);
    • The test does not fail if RAMDirectory is used instead of MockDirectoryWrapper
    • It is quite possible that the problem is with MockDirectoryWrapper or one of its subcomponents, although the locking behavior of IndexWriter is also suspicious
  5. Lucene.Net.Classification.KNearestNeighborClassifierTest::TestPerformance()
    • Happens on netcoreapp2.1 on Windows
    • Happens on netcoreapp1.0 on Windows
    • Happens on net451 on Windows
  6. Lucene.Net.Analysis.NGram.EdgeNGramTokenizerTest::TestFullUTF8Range()
    • Happens on netcoreapp1.0 on Linux
  7. Lucene.Net.Analysis.NGram.NGramTokenizerTest::TestFullUTF8Range()
    • Happens on net451 on Windows
    • Happens on netcoreapp1.0 on Linux
    • Happens on netcoreapp1.0 on Windows
    • Happens on netcoreapp1.0 on macOS
    • Happens on netcoreapp2.1 on macOS
  8. Lucene.Net.Search.VectorHighlight.SimpleFragmentsBuilderTest::TestRandomDiscreteMultiValueHighlighting()
    • Happens on Windows (I think on all target frameworks, but netcoreapp1.0 for sure)
    • Not related to current culture
    • Definitely a problem with the highlighter, not the test or test framework
  9. Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper::TestDateRange()
    • Does not happen on Windows, but happens on both Linux and macOS
    • Related to current culture - fails with new CultureInfo("ar") specifically (and others)
    • Stack trace
      System.FormatException : String '1‏‏/1‏‏/2002' was not recognized as a valid DateTime.
       at System.DateTimeParse.Parse(ReadOnlySpan`1 s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
       at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.AssertDateRangeQueryEquals(StandardQueryParser qp, String field, String startDate, String endDate, DateTime endDateInclusive, Resolution resolution) in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line 837
       at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.TestDateRange() in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line 826
  10. Lucene.Net.Expressions.TestExpressionSorts::TestQueries()
    • Happens on net451 on Windows, x86, in Release mode only (if Release, x86, or different target framework, does not fail)
    • Does not seem to fail using dotnet vstest, only fails in Visual Studio 2017/2019

Note we now have an azure-pipelines.yml configuration file in our repository that anyone can use to setup a build pipeline to see the tests run on Windows, macOS and Linux by setting up a free Azure DevOps account. If you create a public project to run the tests in, it will take roughly an hour to see the test results (a private project will take significantly longer on the free subscription because they only provide a single parallel job).

NOTES

Also, let us know if you find any failing test that is not on this list (aside from the "file in use by another process" issue that is known to happen with multiple tests on Linux and macOS - see LUCENENET-618 if you want to take a crack at that issue)

JIRA link - [LUCENENET-619] created by nightowl888