search

apache / lucenenet

Apache Lucene.NET
https://lucenenet.apache.org/
Apache License 2.0
2.25k stars 639 forks source link

Task: Investigate failing QueryParser Flexible tests (TestNumericQueryParser.cs) #846

Open NightOwl888 opened 1 year ago

NightOwl888 commented 1 year ago

This appears to be a rare edge case, as it hasn't come up before. But then, we have only recently added tests for .NET 7.0. These tests failed on ubuntu-latest (Ubuntu 22.04) using net7.0. Since they failed as a group, they likely have the same underlying cause.

https://dev.azure.com/lucene-net-temp4/Main/_build/results?buildId=232&view=ms.vss-test-web.build-test-results-tab

Note the above link will be cleaned up within 1 month, so the details have been copied below.

TestInclusiveLowerNumericRange

Details

Click to expand ### Error Message Expected: 1, Actual: 0 Query <+INT32:{"-608201030" TO "0"] +INT64:{"-9004819226736869376" TO "0"] +SINGLE:{"-0.49551165" TO "0"] +DOUBLE:{"-0.4581616751646118" TO "0"] +date:{"1908/9/28 5:02:10 -6 西元 10 -06:00 1908" TO "1969/12/31 7:00:00 -5 西元 0 -05:00 1969"]> retrieved 0 document(s), 1 document(s) expected. To reproduce this test result: Option 1: Apply the following assembly-level attributes: ```c# [assembly: Lucene.Net.Util.RandomSeed("0x1aea3b735c59df90")] [assembly: NUnit.Framework.SetCulture("ff-Latn-GM")] ``` Option 2: Use the following .runsettings file: ```xml ``` See the .runsettings documentation at: https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file. ### Stack Trace [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestQuery(String queryStr, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 651](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=651&lineEnd=652&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.assertRangeQuery(Nullable`1 lowerType, Nullable`1 upperType, Boolean lowerInclusive, Boolean upperInclusive, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 568](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=568&lineEnd=569&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestInclusiveLowerNumericRange() in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 427](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=427&lineEnd=428&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)

TestOpenRangeNumericQuery

Details

Click to expand ### Error Message Expected: 2, Actual: 1 Query <+INT32<="0" +INT64<="0" +SINGLE<="0" +DOUBLE<="0" +date<="1969/12/31 7:00:00 -5 西元 0 -05:00 1969"> retrieved 1 document(s), 2 document(s) expected. To reproduce this test result: Option 1: Apply the following assembly-level attributes: ```c# [assembly: Lucene.Net.Util.RandomSeed("0x1aea3b735c59df90")] [assembly: NUnit.Framework.SetCulture("ff-Latn-GM")] ``` Option 2: Use the following .runsettings file: ```xml ``` See the .runsettings documentation at: https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file. ### Stack Trace [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestQuery(String queryStr, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 651](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=651&lineEnd=652&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.assertOpenRangeQuery(NumberType boundType, String operator, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 601](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=601&lineEnd=602&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestOpenRangeNumericQuery() in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 461](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=461&lineEnd=462&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)

TestInclusiveNumericRange

Details

Click to expand ### Error Message Expected: 1, Actual: 0 Query <+INT32:["0" TO "0"] +INT64:["0" TO "0"] +SINGLE:["0" TO "0"] +DOUBLE:["0" TO "0"] +date:["1969/12/31 7:00:00 -5 西元 0 -05:00 1969" TO "1969/12/31 7:00:00 -5 西元 0 -05:00 1969"]> retrieved 0 document(s), 1 document(s) expected. To reproduce this test result: Option 1: Apply the following assembly-level attributes: ```c# [assembly: Lucene.Net.Util.RandomSeed("0x1aea3b735c59df90")] [assembly: NUnit.Framework.SetCulture("ff-Latn-GM")] ``` Option 2: Use the following .runsettings file: ```xml ``` See the .runsettings documentation at: https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file. ### Stack Trace [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestQuery(String queryStr, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 651](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=651&lineEnd=652&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.assertRangeQuery(Nullable`1 lowerType, Nullable`1 upperType, Boolean lowerInclusive, Boolean upperInclusive, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 568](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=568&lineEnd=569&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestInclusiveNumericRange() in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 415](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=415&lineEnd=416&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)

TestSimpleNumericQuery

Details

Click to expand ### Error Message Expected: 1, Actual: 0 Query <+INT32:"0" +INT64:"0" +SINGLE:"0" +DOUBLE:"0" +date:"1969/12/31 7:00:00 -5 西元 0 -05:00 1969"> retrieved 0 document(s), 1 document(s) expected. To reproduce this test result: Option 1: Apply the following assembly-level attributes: ```c# [assembly: Lucene.Net.Util.RandomSeed("0x1aea3b735c59df90")] [assembly: NUnit.Framework.SetCulture("ff-Latn-GM")] ``` Option 2: Use the following .runsettings file: ```xml ``` See the .runsettings documentation at: https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file. ### Stack Trace [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestQuery(String queryStr, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 651](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=651&lineEnd=652&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.assertSimpleQuery(NumberType numberType, Int32 expectedDocCount) in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 633](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=633&lineEnd=634&lineStartColumn=1&lineEndColumn=1&lineStyle=plain) [at Lucene.Net.QueryParsers.Flexible.Standard.TestNumericQueryParser.TestSimpleNumericQuery() in /_/src/Lucene.Net.Tests.QueryParser/Flexible/Standard/TestNumericQueryParser.cs:line 495](https://dev.azure.com/lucene-net-temp4/Main/_git/fdfefc9a-2f87-4b6c-b69a-6efa0c048afe?path=%2F_%2Fsrc%2FLucene.Net.Tests.QueryParser%2FFlexible%2FStandard%2FTestNumericQueryParser.cs&version=GBfix%2FGH-843&_a=contents&line=495&lineEnd=496&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
NightOwl888 commented 7 months ago

@paulirwin - I got another failure on this today on macOS and net7.0. Since you have a mac, think you can take a look?

https://dev.azure.com/lucene-net-temp4/Main/_build/results?buildId=267&view=ms.vss-test-web.build-test-results-tab

[assembly: Lucene.Net.Util.RandomSeed("0xfcd3b1a5ebdbe7c0")]
[assembly: NUnit.Framework.SetCulture("ks-Arab-IN")]

I don't think this has ever failed on Windows. It possibly has something to do with either the recent time zone changes to DateTools (#853) or the upgrade of the TimeZoneConverter NuGet package that we use in tests.

paulirwin commented 7 months ago

@NightOwl888 Unfortunately for our diagnostics, these tests passed on my Mac with that seed/culture set and net7.0:

i.e. TestInclusiveLowerNumericRange:

RandomSeed: 0xfcd3b1a5ebdbe7c0
Culture: ks-Arab-IN
Time Zone: (UTC+12:00) Marshall Islands Time
Default Codec: Lucene40 (Lucene40RWCodec)
Default Similarity: DefaultSimilarity
Nightly: False
Weekly: False
Slow: True
Awaits Fix: False
Directory: random
Verbose: True
Random Multiplier: 1
Parsing: +INT32:{"-2096763321" TO "0"] +INT64:{"-1081678821072489728" TO "0"] +SINGLE:{"-0,046893597" TO "0"] +DOUBLE:{"-0,05523188455414629" TO "0"] +date:{"1932 de julhet 3 17:06:43 CE 43 +07:00 1932" TO "1970 de genièr 1 07:00:00 CE 0 +07:00 1970"]
Querying: +INT32:{‎-‎2096763321 TO 0] +INT64:{‎-‎1081678821072489728 TO 0] +SINGLE:{‎-‎0٫046893597 TO 0] +DOUBLE:{‎-‎0٫05523188455414629 TO 0] +date:{‎-‎1183297997000 TO 0]
Query <+INT32:{"-2096763321" TO "0"] +INT64:{"-1081678821072489728" TO "0"] +SINGLE:{"-0,046893597" TO "0"] +DOUBLE:{"-0,05523188455414629" TO "0"] +date:{"1932 de julhet 3 17:06:43 CE 43 +07:00 1932" TO "1970 de genièr 1 07:00:00 CE 0 +07:00 1970"]> retrieved 1 document(s), 1 document(s) expected.
Parsing: +INT32:{"0" TO "2096763321"] +INT64:{"0" TO "1081678821072489728"] +SINGLE:{"0" TO "0,046893597"] +DOUBLE:{"0" TO "0,05523188455414629"] +date:{"1970 de genièr 1 07:00:00 CE 0 +07:00 1970" TO "2007 de julhet 1 22:53:17 CE 17 +09:00 2007"]
Querying: +INT32:{0 TO 2096763321] +INT64:{0 TO 1081678821072489728] +SINGLE:{0 TO 0٫046893597] +DOUBLE:{0 TO 0٫05523188455414629] +date:{0 TO 1183297997000]
Query <+INT32:{"0" TO "2096763321"] +INT64:{"0" TO "1081678821072489728"] +SINGLE:{"0" TO "0,046893597"] +DOUBLE:{"0" TO "0,05523188455414629"] +date:{"1970 de genièr 1 07:00:00 CE 0 +07:00 1970" TO "2007 de julhet 1 22:53:17 CE 17 +09:00 2007"]> retrieved 1 document(s), 1 document(s) expected.
Parsing: +INT32:{"-2096763321" TO "2096763321"] +INT64:{"-1081678821072489728" TO "1081678821072489728"] +SINGLE:{"-0,046893597" TO "0,046893597"] +DOUBLE:{"-0,05523188455414629" TO "0,05523188455414629"] +date:{"1932 de julhet 3 17:06:43 CE 43 +07:00 1932" TO "2007 de julhet 1 22:53:17 CE 17 +09:00 2007"]
Querying: +INT32:{‎-‎2096763321 TO 2096763321] +INT64:{‎-‎1081678821072489728 TO 1081678821072489728] +SINGLE:{‎-‎0٫046893597 TO 0٫046893597] +DOUBLE:{‎-‎0٫05523188455414629 TO 0٫05523188455414629] +date:{‎-‎1183297997000 TO 1183297997000]
Query <+INT32:{"-2096763321" TO "2096763321"] +INT64:{"-1081678821072489728" TO "1081678821072489728"] +SINGLE:{"-0,046893597" TO "0,046893597"] +DOUBLE:{"-0,05523188455414629" TO "0,05523188455414629"] +date:{"1932 de julhet 3 17:06:43 CE 43 +07:00 1932" TO "2007 de julhet 1 22:53:17 CE 17 +09:00 2007"]> retrieved 2 document(s), 2 document(s) expected.
Parsing: +INT32:{"-2096763321" TO "-2096763321"] +INT64:{"-1081678821072489728" TO "-1081678821072489728"] +SINGLE:{"-0,046893597" TO "-0,046893597"] +DOUBLE:{"-0,05523188455414629" TO "-0,05523188455414629"] +date:{"1932 de julhet 3 17:06:43 CE 43 +07:00 1932" TO "1932 de julhet 3 17:06:43 CE 43 +07:00 1932"]
Querying: +INT32:{‎-‎2096763321 TO ‎-‎2096763321] +INT64:{‎-‎1081678821072489728 TO ‎-‎1081678821072489728] +SINGLE:{‎-‎0٫046893597 TO ‎-‎0٫046893597] +DOUBLE:{‎-‎0٫05523188455414629 TO ‎-‎0٫05523188455414629] +date:{‎-‎1183297997000 TO ‎-‎1183297997000]
Query <+INT32:{"-2096763321" TO "-2096763321"] +INT64:{"-1081678821072489728" TO "-1081678821072489728"] +SINGLE:{"-0,046893597" TO "-0,046893597"] +DOUBLE:{"-0,05523188455414629" TO "-0,05523188455414629"] +date:{"1932 de julhet 3 17:06:43 CE 43 +07:00 1932" TO "1932 de julhet 3 17:06:43 CE 43 +07:00 1932"]> retrieved 0 document(s), 0 document(s) expected.

As an aside: the documented lucene.testSettings.config approach doesn't seem to work at least as documented on my Mac. I haven't yet dug into why, but I created that file in the repo root (which should be "between the executable and the root of the drive") and it didn't take the culture/seed. Instead I added those attributes as you provided to the QueryParser tests AssemblyInfo.cs directly. Let me know if there's a better way to do that. Thanks!

NightOwl888 commented 7 months ago

Yeah, that is what happened when I tried chasing this down on Ubuntu. It seems that this is due to the exact system configuration that the test is running in. It may be easier just to revert those 2 time zone changes one at a time to see if the error stops happening, although it takes lots of runs to get a failure.

lucene.testSettings.config - maybe try the name all lowercase?

Instead I added those attributes as you provided to the QueryParser tests AssemblyInfo.cs directly. Let me know if there's a better way to do that. Thanks!

The attribute is what I usually use because it overrides any other config on the system.

It is possible to specify system properties in a .runsettings file.

Use the following .runsettings file:

<RunSettings>
  <TestRunParameters>
    <Parameter name="tests:seed" value="0xfcd3b1a5ebdbe7c0" />
    <Parameter name="tests:culture" value="ks-Arab-IN" />
  </TestRunParameters>
</RunSettings>
See the .runsettings documentation at: https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file.

It is also possible to pass them via command line, but the syntax is pretty weird and varies by shell: https://github.com/Microsoft/vstest-docs/blob/main/docs/RunSettingsArguments.md. There is an example of this being done in TestIndexWriterOnJRECrash.

NightOwl888 commented 1 week ago

I got another failure. This time on Linux: https://dev.azure.com/shad0962/Experiments/_build/results?buildId=2528&view=ms.vss-test-web.build-test-results-tab&runId=1006431&resultId=100248&paneView=debug.

Reposing the repeat data here, since the CI environment will eventually be cleaned up.

Expected: 1, Actual: 0

Query <+INT32:{"0" TO "554807661"] +INT64:{"0" TO "4750245968736588800"] +SINGLE:{"0" TO "0.55612606"] +DOUBLE:{"0" TO "0.27287609303204485"] +date:{"1970年1月1日 星期四 4:00 西元 0 +04:00 1970" TO "1993年9月30日 星期四 9:15 西元 14 +04:00 1993"]> retrieved 0 document(s), 1 document(s) expected.

To reproduce this test result:

Option 1:

Apply the following assembly-level attributes:

[assembly: Lucene.Net.Util.RandomSeed("0xb19ea259d1e18b66")]
[assembly: NUnit.Framework.SetCulture("es-BZ")]

Option 2:

Use the following .runsettings file:

<RunSettings>
  <TestRunParameters>
    <Parameter name="tests:seed" value="0xb19ea259d1e18b66" />
    <Parameter name="tests:culture" value="es-BZ" />
  </TestRunParameters>
</RunSettings>
Option 3:

Create the following lucene.testsettings.json file somewhere between the test assembly and the root of your drive:

{
"tests": {
"seed": "0xb19ea259d1e18b66",
"culture": "es-BZ"
}
}