search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.29k stars 4.74k forks source link

Test failure System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) #60753

Closed VincentBu closed 2 years ago

VincentBu commented 3 years ago

Run: runtime-libraries-coreclr outerloop 20211021.5

Failed test:

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x86-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

One of error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 442

or

Assert.True() Failure
Expected: True
Actual:   False

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 436
ghost commented 3 years ago

Tagging subscribers to this area: @eerhardt, @dotnet/area-system-text-regularexpressions See info in area-owners.md if you want to be subscribed.

Issue Details
Run: [runtime-libraries-coreclr outerloop 20211021.5](https://dev.azure.com/dnceng/public/_build/results?buildId=1434109&view=ms.vss-test-web.build-test-results-tab&runId=41420594&resultId=104208&paneView=debug) Failed test: ``` net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) net7.0-windows-Release-x64-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504 - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) net7.0-windows-Release-x86-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None) - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380 - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open - System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) ``` **One of error message:** ``` Assert.NotEqual() Failure Expected: Not 'ᲀ' Actual: 'ᲀ' Stack trace at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 442 ``` or ``` Assert.True() Failure Expected: True Actual: False Stack trace at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 436 ```
Author: VincentBu
Assignees: -
Labels: `arch-arm32`, `arch-arm64`, `arch-x86`, `area-System.Text.RegularExpressions`, `os-linux`, `os-mac-os-x`, `os-windows`, `arch-x64`
Milestone: -
stephentoub commented 3 years ago

I believe the issue here is different data coming from either NLS or older ICUs. I can repro the same failures by setting the environment variable: DOTNET_SYSTEM_GLOBALIZATION_USENLS=1 and then running outerloop unit tests.

@veanes, @olsaarik, I think this highlights we'll need to revisit the pre-generation of the relation data (for reasons beyond the size ones we've already discussed). The data can vary based on the machine on which pregeneration was performed, but we ship a single S.T.RegularExpressions.dll binary that's used regardless of OS version and regardless of what globalization library is in use for a given process. For both size and correctness/consistency then, we'll likely need to remove the pregenerated data and come up with a scheme for computing it lazily at run-time.

cc: @danmoseley

danmoseley commented 3 years ago

remove the pregenerated data and come up with a scheme for computing it lazily at run-time.

Same for the source generator? Basically our position on culture is always: we respect the thread's culture at the time the regex object is first used.

stephentoub commented 3 years ago

Same for the source generator?

It's a question of where are we baking the assumptions into. I think it's reasonable to bake them into the user assembly being built. I think it's unreasonable to bake them into S.T.RE.dll.

veanes commented 3 years ago

The generated tables really depend on the definitions/semantics of Unicode standard categories and in this case IgnoreCase rules for 'i'. I was under the impression that these are fixed for a given system version. The 'i' part can be tested and generated locally for a given runtime also if there are differences with handling of Turkish 'i' -- I'm sure that's what this is about.

veanes commented 3 years ago

Also, if it is ever possible that the Unicode Categories actually change during the lifetime of one .NET version (thus actually affecting the semantics of Regex matching, which I think I've seen happen once), then the generated BDD tables need to be updated simultaneously -- or generated dynamically.

stephentoub commented 3 years ago

I was under the impression that these are fixed for a given system version.

We get data from the underlying OS and/or ICU library in play. You can see the effect of this by running this on .NET 6:

using System.Globalization;

var enUs = new CultureInfo("en-US");
var invariant = CultureInfo.InvariantCulture;

for (int i = 0; i <= char.MaxValue; i++)
{
    if (char.ToUpper((char)i, enUs) != char.ToUpper((char)i, invariant))
    {
        Console.WriteLine($"ToUpper: {i}");
    }
    if (char.ToLower((char)i, enUs) != char.ToLower((char)i, invariant))
    {
        Console.WriteLine($"ToLower: {i}");
    }
}

You should see output like:

ToLower: 304
ToUpper: 305

Now run it again, but this time with this environment variable set:

$env:DOTNET_SYSTEM_GLOBALIZATION_USENLS=1

Now you should see output more like this:

ToUpper: 181
ToLower: 304
ToUpper: 305
ToUpper: 383
ToUpper: 453
ToLower: 453
ToUpper: 456
ToLower: 456
ToUpper: 459
ToLower: 459
ToUpper: 498
ToLower: 498
ToUpper: 837
ToUpper: 962
ToUpper: 976
ToUpper: 977
ToLower: 978
ToLower: 979
ToLower: 980
ToUpper: 981
ToUpper: 982
ToUpper: 1008
ToUpper: 1009
ToLower: 1012
ToUpper: 1013
ToUpper: 7835
ToLower: 7838
ToUpper: 8126
ToLower: 8486
ToLower: 8490
ToLower: 8491
danmoseley commented 3 years ago

To clarify what Stephen said it's not just that flag that affects the behavior but whatever libicu version is present on that OS or distro. Each version has slightly different data, although probably casing is relatively stable.

VincentBu commented 3 years ago

Failed again in: runtime-libraries-coreclr outerloop 20211031.1

Test failure:

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-SLES.12.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 380

Failed test:

net7.0-Linux-Release-x64-CoreCLR_release-(Mariner.1.0.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-1.0-helix-20210528192219-92bf620
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210913123654-4f64125
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20210304164428-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.314.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.313.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-amd64-20210910135845-8a6f4f3
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message：

Assert.True() Failure
Expected: True
Actual:   False

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 436
VincentBu commented 3 years ago

Failed again in: runtime-libraries-coreclr outerloop 20211108.1

Failed test:

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 382

Failed test:

net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210913123654-4f64125
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20210304164428-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Mariner.1.0.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-1.0-helix-20210528192219-92bf620
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.314.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.313.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-amd64-20210910135845-8a6f4f3
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.True() Failure
Expected: True
Actual:   False

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 438
VincentBu commented 2 years ago

Failed again in: runtime-libraries-coreclr outerloop 20211116.1

Failed test:

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-SLES.12.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: Compiled)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(options: None)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error mesage:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 382

Failed test:

net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210913123654-4f64125
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20210304164428-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Mariner.1.0.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-1.0-helix-20210528192219-92bf620
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.314.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.313.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-amd64-20210910135845-8a6f4f3
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.True() Failure
Expected: True
Actual:   False

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 438
VincentBu commented 2 years ago

Failed again in: runtime-libraries-coreclr outerloop 20211124.6

Failed test:

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x64-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 420

Failed test:

net7.0-Linux-Release-x64-CoreCLR_release-(Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20210304164428-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Mariner.1.0.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-1.0-helix-20210528192219-92bf620
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.314.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.313.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-amd64-20210910135845-8a6f4f3
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-SLES.12.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.True() Failure
Expected: True
Actual:   False

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 414
buyaa-n commented 2 years ago

Failed again https://github.com/dotnet/runtime/pull/62095/checks?check_run_id=4401635475

    System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs(444,0): at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options)
    System.Text.RegularExpressions.Tests.RegexMatchTests.Match_Timeout_Loop_Throws [SKIP]
      Condition(s) not met: "IsWindows"
  Finished:    System.Text.RegularExpressions.Tests
=== TEST EXECUTION SUMMARY ===
   System.Text.RegularExpressions.Tests  Total: 825, Errors: 0, Failed: 1, Skipped: 2, Time: 85.378s

https://github.com/dotnet/runtime/pull/62095/checks?check_run_id=4401635312

    System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking) [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs(444,0): at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options)
  Finished:    System.Text.RegularExpressions.Tests
=== TEST EXECUTION SUMMARY ===
   System.Text.RegularExpressions.Tests  Total: 825, Errors: 0, Failed: 1, Skipped: 2, Time: 44.935s
VincentBu commented 2 years ago

Failed again in: runtime-libraries-coreclr outerloop 20211202.6

Failed test:

net7.0-Linux-Release-x64-CoreCLR_release-SLES.12.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-SLES.12.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x64-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x86-CoreCLR_release-Windows.10.Amd64.ServerRS5.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 420

Failed test:

net7.0-OSX-Release-arm64-CoreCLR_release-OSX.1100.ARM64.Open
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Mariner.1.0.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-1.0-helix-20210528192219-92bf620
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20210304164428-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210913123654-4f64125
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm-CoreCLR_release-(Debian.11.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm32v7-20210304164347-5a7c380
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.314.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-(Alpine.313.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-helix-amd64-20210910135845-8a6f4f3
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.True() Failure
Expected: True
Actual:   False

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/RegexCultureTests.cs:line 414
joperezr commented 2 years ago

This doesn't seem fixed by #67184 particularly because this is a problem caused by NLS vs ICU which is still an issue we have even our IgnoreCase overhaul. This should get fixed once https://github.com/dotnet/runtime/issues/67624 gets fixed as they are related.

VincentBu commented 2 years ago

Failed again in: runtime-libraries-coreclr outerloop 20220410.7

Failed test:

net7.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexCultureTests.cs:line 429
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr)
VincentBu commented 2 years ago

Failed again in: runtime-libraries-coreclr outerloop 20220418.8

Failed test:

net7.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(RegexOptions options) in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexCultureTests.cs:line 429
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.CoreCLR.cs:line 379
VincentBu commented 2 years ago

Failed again in: runtime-libraries-coreclr outerloop 20220426.3

Failed test:

net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
net7.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-Linux-Release-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-x64-CoreCLR_release-RedHat.7.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)

net7.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673

- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelationBorderCasesInNonBacktracking(options: NonBacktracking)
- System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation

Error message:

Assert.NotEqual() Failure
Expected: Not 'ᲀ'
Actual:   'ᲀ'

Stack trace
   at System.Text.RegularExpressions.Tests.RegexCultureTests.TestIgnoreCaseRelation() in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexCultureTests.cs:line 367
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.CoreCLR.cs:line 387
danmoseley commented 2 years ago

@joperezr looks like #67624 didn't fix this either.