search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.48k stars 4.76k forks source link

System.Text.RegularExpressions.Tests JitMinOpts only timeout #10680

Closed BruceForstall closed 3 years ago

BruceForstall commented 6 years ago 
----- start 19:14:03.62 ===============  To repro directly: ===================================================== 
pushd C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64\
set COMPlus_TieredCompilation=0 
call C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm64\dotnet.exe xunit.console.netcore.exe System.Text.RegularExpressions.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================

C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64>set COMPlus_TieredCompilation=0  

C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64>set COMPlus_JITMinOpts=1 

C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64>call C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm64\dotnet.exe xunit.console.netcore.exe System.Text.RegularExpressions.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
xUnit.net console test runner (64-bit .NET Core)
Copyright (C) 2014 Outercurve Foundation.

Discovering: System.Text.RegularExpressions.Tests
Discovered:  System.Text.RegularExpressions.Tests
Starting:    System.Text.RegularExpressions.Tests
   System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix [FAIL]
      Timed out after 60000ms waiting for remote process 5512
      Expected: True
      Actual:   False
      Stack Trace:
         D:\j\workspace\arm64_cross_c---101bec61\_\fx\src\CoreFx.Private.TestUtilities\src\System\Diagnostics\RemoteExecutorTestBase.cs(308,0): at System.Diagnostics.RemoteExecutorTestBase.RemoteInvokeHandle.Dispose(Boolean disposing)
         D:\j\workspace\arm64_cross_c---101bec61\_\fx\src\CoreFx.Private.TestUtilities\src\System\Diagnostics\RemoteExecutorTestBase.cs(295,0): at System.Diagnostics.RemoteExecutorTestBase.RemoteInvokeHandle.Dispose()
         D:\j\workspace\arm64_cross_c---101bec61\_\fx\src\System.Text.RegularExpressions\tests\Regex.Match.Tests.cs(795,0): at System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix()
Finished:    System.Text.RegularExpressions.Tests

=== TEST EXECUTION SUMMARY ===
   System.Text.RegularExpressions.Tests  Total: 1498, Errors: 0, Failed: 1, Skipped: 0, Time: 197.040s
----- end 19:17:36.82 ----- exit code 1 ----------------------------------------------------------

Related: https://github.com/dotnet/coreclr/issues/17754

category:testing theme:minopts skill-level:intermediate cost:medium

kouvel commented 6 years ago

This is also failing on x64 on several OSes in the coreclr repo's corefx test runs when tiering is enabled (equivalent to minopts for this test). Not sure if it's related to the arm64 failure, but it's not clear to me what the problem is on x64. Rough numbers from my machine:

corefx repo all regex tests -

just the Match_ExcessPrefix test (times in ms for each regex in test) -

I don't see any numbers close to 60 seconds and yet it's timing out. I noticed that if I add RegexOptions.Compiled, the last regex in the test is taking ~60000 ms to complete with release coreclr and release corefx (and 1000 ms with debug coreclr and debug corefx), but that option is not specified in the test.

kouvel commented 6 years ago

PR to temporarily disable test: https://github.com/dotnet/corefx/pull/31500

echesakov commented 6 years ago

I investigated this timeout failure and I believe this is due to the fact that under debug and minopts JIT does not expand CORINFO_INTRINSIC_StringGetChar and instead emit call to COMString::GetCharAt helper in System.Text.RegularExpressions.RegexBoyerMoore:.ctor(ref,bool,bool,ref):this.

I did simple experiment and for optimized code I disabled ONLY expansion of CORINFO_INTRINSIC_StringGetChar in importer which brings up running time for System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix from 2s up to 550s (which is roughly the same time needed for the test to finish under JITMinOpts).

Another experiment showed that setting JitMinOptsName=System.Text.RegularExpressions.RegexBoyerMoore:.ctor(ref,bool,bool,ref):this is enough for this test to fail due to timeout out.

@dotnet/jit-contrib Is this supposed to be this way? This call to COMString::GetCharAt happens in a loop and according to PerfView takes 99.4 Inc %

I also attached JitDisasm in case if someone is interested to see what's going on.

optimized code dasm.txt optimized code - forbid CORINFO_INTRINSIC_StringGetChar expansion dasm.txt minopts dasm.txt

BruceForstall commented 6 years ago

The JIT wants to leave calls as calls, and not do the intrinsic optimization, during MinOpts. This seems completely reasonable. The test does pass, it's just super slow. That would be expected. Seems like we just need a way to bump the timeout of this test, significantly, under Minopts/debuggable code.

AndyAyersMS commented 6 years ago

This is perhaps and example where Tier0 should diverge from minopts?

kouvel commented 6 years ago

Is that an issue even when RegexOptions.Compiled is not specified? The test doesn't specify RegexOptions.Compiled and locally the test was running fairly quickly with debug coreclr and debug corefx. Only when I change the test to use RegexOptions.Compiled it was very slow, any idea why it would be failing in the CI?

kouvel commented 6 years ago

The test was disabled for the coreclr repo's "CoreFX Tests" job in https://github.com/dotnet/coreclr/pull/19488, once this is fixed please re-enable the test.

danmoseley commented 6 years ago

@ViktorHofer fyi.

stephentoub commented 4 years ago

FYI, this test has been changed since this issue was opened.

am11 commented 3 years ago

With engine SourceGenerated, same test has started to fail:

windows x64 - https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-1846899be12742d186/System.Text.RegularExpressions.Tests/1/console.a2979f90.log

    System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated) [FAIL]
      Half-way through waiting for remote process.
      Memory load: 11
      ...
      Timed out at 9/22/2021 11:53:49 PM after 60000ms waiting for remote process.
      Wrote mini dump to: C:\h\w\99FC08C0\w\B18309D5\uploads\5588.fckw5zf0.h4d.dmp
        Process ID: 5588
        Handle: 1984
        Name: dotnet
        MainModule: C:\h\w\99FC08C0\p\dotnet.exe
        StartTime: 9/22/2021 11:52:49 PM
        TotalProcessorTime: 00:01:02.6718750

linux musl arm64 - https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-ae9baf50200a4cdebf/System.Text.RegularExpressions.Tests/1/console.28be79eb.log

    System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated) [FAIL]
      Half-way through waiting for remote process.
      Timed out at 9/23/2021 12:13:52 AM after 60000ms waiting for remote process.
        Process ID: 259
        Handle: 3160
        Name: dotnet
        MainModule: /root/helix/work/correlation/dotnet
        StartTime: 9/23/2021 12:12:51 AM
        TotalProcessorTime: 00:01:03.2400000
stephentoub commented 3 years ago

With engine SourceGenerated, same test has started to fail:

I'd expect this to behave similarly to RegexOptions.Compiled, but with even more expense, as it's also running Roslyn to compile C#.

You're seeing this failure only in JitMinOpts, or in general?

VincentBu commented 3 years ago

Failed again in: runtime 20210922.59

Failed test:

net7.0-windows-Release-x64-CoreCLR_checked-Windows.10.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated)

net7.0-windows-Release-x86-CoreCLR_checked-Windows.10.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated)

Error message:

Half-way through waiting for remote process.
Memory load: 12

Image Name                     PID Services

========================= ======== ============================================
svchost.exe                    772 BrokerInfrastructure, DcomLaunch, LSM,

PlugPlay, Power, SystemEventsBroker

svchost.exe                    836 RpcEptMapper, RpcSs

svchost.exe                    944 Appinfo, CertPropSvc, gpsvc, iphlpsvc,

lfsvc, ProfSvc, sacsvr, Schedule, SENS,

SessionEnv, ShellHWDetection, Themes,

UserManager, Winmgmt, WpnService, wuauserv

svchost.exe                    952 TermService

svchost.exe                   1012 Dhcp, EventLog, lmhosts, TimeBrokerSvc,

vmictimesync

svchost.exe                    444 NcbService, PcaSvc, TrkWks, UALSVC,

UmRdpService, vmickvpexchange,

vmicshutdown, WdiSystemHost

svchost.exe                     84 CDPSvc, EventSystem, FontCache, netprofm,

nsi, W32Time, WinHttpAutoProxySvc

svchost.exe                   1140 CryptSvc, Dnscache, LanmanWorkstation,

NlaSvc, WinRM

svchost.exe                   1180 vmicheartbeat

svchost.exe                   1276 BFE, CoreMessagingRegistrar, DPS, MpsSvc,

pla

svchost.exe                   1484 Wcmsvc

svchost.exe                   2028 LanmanServer

svchost.exe                   1320 DiagTrack

svchost.exe                   1512 StateRepository, tiledatamodelsvc

svchost.exe                   1696 AppHostSvc

svchost.exe                   1904 W3SVC, WAS

svchost.exe                   3152 CDPUserSvc_43fee, OneSyncSvc_43fee

Timed out at 9/22/2021 9:32:11 PM after 60000ms waiting for remote process.
Wrote mini dump to: C:\h\w\B8C009E0\w\C4A90A8F\uploads\4664.i4er4p0z.55v.dmp
Process ID: 4664
Handle: 1496
Name: dotnet
MainModule: C:\h\w\B8C009E0\p\dotnet.exe
StartTime: 9/22/2021 9:31:11 PM
TotalProcessorTime: 00:01:02.5312500
Threads:
Thread #1 (OS 0x39C)   [MTA]
[HelperMethodFrame_1OBJ] (System.Threading.Monitor.ObjWait)
System.Threading.Monitor.Wait(System.Object, Int32)
System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken)
System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken)
System.Threading.Tasks.Task.InternalWaitCore(Int32, System.Threading.CancellationToken)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.WaitForWorkers()
Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethodBodies(Microsoft.CodeAnalysis.CSharp.CSharpCompilation, Microsoft.CodeAnalysis.CSharp.Emit.PEModuleBuilder, Boolean, Boolean, Boolean, Boolean, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, System.Predicate1<Microsoft.CodeAnalysis.CSharp.Symbol>, System.Threading.CancellationToken)          Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CompileMethods(Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder, Boolean, Boolean, Boolean, Microsoft.CodeAnalysis.DiagnosticBag, System.Predicate1<Microsoft.CodeAnalysis.Symbols.ISymbolInternal>, System.Threading.CancellationToken)
Microsoft.CodeAnalysis.Compilation.Emit(System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.ResourceDescription>, Microsoft.CodeAnalysis.Emit.EmitOptions, Microsoft.CodeAnalysis.IMethodSymbol, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.EmbeddedText>, Microsoft.CodeAnalysis.RebuildData, Microsoft.CodeAnalysis.CodeGen.CompilationTestData, System.Threading.CancellationToken)
Microsoft.CodeAnalysis.Compilation.Emit(System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.ResourceDescription>, Microsoft.CodeAnalysis.Emit.EmitOptions, Microsoft.CodeAnalysis.IMethodSymbol, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.EmbeddedText>, System.IO.Stream, Microsoft.CodeAnalysis.RebuildData, System.Threading.CancellationToken)
Microsoft.CodeAnalysis.Compilation.Emit(System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.ResourceDescription>, Microsoft.CodeAnalysis.Emit.EmitOptions, Microsoft.CodeAnalysis.IMethodSymbol, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.EmbeddedText>, System.IO.Stream, System.Threading.CancellationToken)
System.Text.RegularExpressions.Tests.RegexGeneratorHelper+<SourceGenRegexAsync>d__6.MoveNext()
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.Text.RegularExpressions.Tests.RegexGeneratorHelper+<SourceGenRegexAsync>d__6, System.Text.RegularExpressions.Tests]](<SourceGenRegexAsync>d__6 ByRef)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder1[[System.__Canon, System.Private.CoreLib]].Start[[System.Text.RegularExpressions.Tests.RegexGeneratorHelper+<SourceGenRegexAsync>d__6, System.Text.RegularExpressions.Tests]](<SourceGenRegexAsync>d__6 ByRef)           System.Text.RegularExpressions.Tests.RegexGeneratorHelper.SourceGenRegexAsync(System.String, System.Nullable1<System.Text.RegularExpressions.RegexOptions>, System.Nullable1<System.TimeSpan>, System.Threading.CancellationToken)          System.Text.RegularExpressions.Tests.RegexHelpers+<GetRegexAsync>d__8.MoveNext()            System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.Text.RegularExpressions.Tests.RegexHelpers+<GetRegexAsync>d__8, System.Text.RegularExpressions.Tests]](<GetRegexAsync>d__8 ByRef)          System.Text.RegularExpressions.Tests.RegexMatchTests+<>c+<<Match_ExcessPrefix>b__29_0>d.MoveNext()          System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.Text.RegularExpressions.Tests.RegexMatchTests+<>c+<<Match_ExcessPrefix>b__29_0>d, System.Text.RegularExpressions.Tests]](<<Match_ExcessPrefix>b__29_0>d ByRef)             System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.Text.RegularExpressions.Tests.RegexMatchTests+<>c+<<Match_ExcessPrefix>b__29_0>d, System.Text.RegularExpressions.Tests]](<<Match_ExcessPrefix>b__29_0>d ByRef)             System.Text.RegularExpressions.Tests.RegexMatchTests+<>c.<Match_ExcessPrefix>b__29_0(System.String)             [DebuggerU2MCatchHandlerFrame]          [HelperMethodFrame_PROTECTOBJ] (System.RuntimeMethodHandle.InvokeMethod)            System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)          System.Reflection.MethodBase.Invoke(System.Object, System.Object[])             Microsoft.DotNet.RemoteExecutor.Program.Main(System.String[])       Thread #2 (OS 0xAE0) [Finalizer] [Background] [MTA]             [DebuggerU2MCatchHandlerFrame]      Thread #3 (OS 0x1540) [Background] [MTA]            [DebuggerU2MCatchHandlerFrame]      Thread #5 (OS 0x115C) [Thread pool worker] [Background] [MTA]           [DebuggerU2MCatchHandlerFrame]      Thread #8 (OS 0x146C) [Thread pool worker] [Background] [MTA]           [InlinedCallFrame]          [InlinedCallFrame]          Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, Int32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32)             System.Threading.LowLevelLifoSemaphore.WaitCore(Int32)          System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32)             System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean)             System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()            System.Threading.Thread.StartCallback()             [DebuggerU2MCatchHandlerFrame]      Thread #9 (OS 0x1320) [Thread pool worker] [Background] [MTA]           [InlinedCallFrame]          [InlinedCallFrame]          Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, Int32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32)             System.Threading.LowLevelLifoSemaphore.WaitCore(Int32)          System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32)             System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean)             System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()            System.Threading.Thread.StartCallback()             [DebuggerU2MCatchHandlerFrame]      Thread #10 (OS 0xE9C) [Thread pool worker] [Background] [MTA]           [PrestubMethodFrame] (Microsoft.CodeAnalysis.MetadataDecoder<System.__Canon,System.__Canon,System.__Canon,System.__Canon,System.__Canon>.DecodeTypeOrThrow)             Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].GetTypeOfTypeSpec(System.Reflection.Metadata.TypeSpecificationHandle)
Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].GetTypeOfToken(System.Reflection.Metadata.EntityHandle, Boolean ByRef)            Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].EnqueueTypeDefInterfacesAndBaseTypeOrThrow(System.Collections.Generic.Queue1<System.Reflection.Metadata.TypeDefinitionHandle>, System.Collections.Generic.Queue1<System.__Canon>, System.Reflection.Metadata.TypeDefinitionHandle)
Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].FindMethodSymbolInSuperType(System.Reflection.Metadata.TypeDefinitionHandle, System.Reflection.Metadata.MethodDefinitionHandle)           Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].GetExplicitlyOverriddenMethods(System.Reflection.Metadata.TypeDefinitionHandle, System.Reflection.Metadata.MethodDefinitionHandle, System.__Canon)
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol.get_ExplicitInterfaceImplementations()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol.ComputeMethodKind()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol.get_MethodKind()
Microsoft.CodeAnalysis.CSharp.Symbols.SymbolExtensions.IsParameterlessConstructor(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol)
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamedTypeSymbol.LoadMembers()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamedTypeSymbol.EnsureAllMembersAreLoaded()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamedTypeSymbol.GetSimpleNonTypeMembers(System.String)
Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol.GetOperators(System.String)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.<ComputeApplicableUserDefinedImplicitConversionSet>g__addCandidatesFromType|133_0(Microsoft.CodeAnalysis.CSharp.Symbols.TypeParameterSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol, Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder1<Microsoft.CodeAnalysis.CSharp.UserDefinedConversionAnalysis>, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef, Boolean)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.ComputeApplicableUserDefinedImplicitConversionSet(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder1<Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol>, Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder1<Microsoft.CodeAnalysis.CSharp.UserDefinedConversionAnalysis>, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef, Boolean)             Microsoft.CodeAnalysis.CSharp.ConversionsBase.AnalyzeImplicitUserDefinedConversions(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.GetImplicitUserDefinedConversion(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef)          Microsoft.CodeAnalysis.CSharp.ConversionsBase.ClassifyImplicitConversionFromExpression(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.ClassifyConversionFromExpressionForCast(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef)             Microsoft.CodeAnalysis.CSharp.ConversionsBase.ClassifyConversionFromExpression(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindCastCore(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeWithAnnotations, Boolean, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindCast(Microsoft.CodeAnalysis.CSharp.Syntax.CastExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindValue(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, BindValueKind)
Microsoft.CodeAnalysis.CSharp.Binder.BindAssignment(Microsoft.CodeAnalysis.CSharp.Syntax.AssignmentExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindValue(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, BindValueKind)
Microsoft.CodeAnalysis.CSharp.Binder.BindRValueWithoutTargetType(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionStatement(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode, Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Boolean, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionStatement(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionStatementSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(Microsoft.CodeAnalysis.CSharp.Syntax.StatementSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindBlockParts(Microsoft.CodeAnalysis.CSharp.Syntax.BlockSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindBlock(Microsoft.CodeAnalysis.CSharp.Syntax.BlockSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(Microsoft.CodeAnalysis.CSharp.Syntax.StatementSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindConstructorBody(Microsoft.CodeAnalysis.CSharp.Syntax.ConstructorDeclarationSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindMethodBody(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.BindMethodBody(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol, Microsoft.CodeAnalysis.CSharp.TypeCompilationState, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, VariableState, Boolean, Microsoft.CodeAnalysis.CSharp.ImportChain ByRef, Boolean ByRef, InitialState ByRef)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethod(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol, Int32, ProcessedFieldInitializers ByRef, Microsoft.CodeAnalysis.CSharp.SynthesizedSubmissionFields, Microsoft.CodeAnalysis.CSharp.TypeCompilationState)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileNamedType(Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol)
Microsoft.CodeAnalysis.CSharp.MethodCompiler+<>c__DisplayClass23_0.<CompileNamedTypeAsync>b__0()
Roslyn.Utilities.UICultureUtilities+<>c__DisplayClass5_0.<WithCurrentUICulture>b__0()
System.Threading.Tasks.Task.InnerInvoke()
System.Threading.Tasks.Task+<>c.<.cctor>b__271_0(System.Object)
System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
System.Threading.ThreadPoolWorkQueue.Dispatch()
System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
System.Threading.Thread.StartCallback()
[DebuggerU2MCatchHandlerFrame]
Thread #11 (OS 0x48C) [Thread pool worker] [Background] [MTA]
[InlinedCallFrame]
[InlinedCallFrame]
Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, Int32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32)
System.Threading.LowLevelLifoSemaphore.WaitCore(Int32)
System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32)
System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean)
System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
System.Threading.Thread.StartCallback()
[DebuggerU2MCatchHandlerFrame]

Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 57
   at System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(RegexEngine engine) in /_/src/libraries/System.Text.RegularExpressions/tests/Regex.Match.Tests.cs:line 1095
am11 commented 3 years ago

You're seeing this failure only in JitMinOpts, or in general?

In general. I noticed this in a few recent outerloop runs on main, also failing in some PRs.

stephentoub commented 3 years ago

Yeah, it looks like running Roslyn on a checked runtime is just too slow.

stephentoub commented 3 years ago

Actually, this test is already disabled against both checked and min opts: https://github.com/dotnet/runtime/blob/00aa6518a6b0a7452af5a76133ee1781054e4c7d/src/libraries/System.Text.RegularExpressions/tests/Regex.Match.Tests.cs#L1091-L1094 But clearly from the output above, it's running on checked.

@safern, any known issue here?

am11 commented 3 years ago

Ah, perhaps SkipOnTargetFramework followed by SkipOnCoreClr is not being handled properly?

Looks like it also failed a few times on arm64-musl release (e.g. https://github.com/dotnet/runtime/runs/3683859860) and windows x64 debug.

am11 commented 3 years ago

Ops, I missed that RuntimeConfiguration == checked, LibrariesConfiguration == Release. Double checked in runfo results: https://github.com/dotnet/runtime/issues/59541, they are all failing on checked.

am11 commented 3 years ago

In SkipOnCoreClrDiscoverer, we use stressMode && runtimeConfiguration: https://github.com/dotnet/arcade/blob/8f58bb36250772fc0a4a0305daf307292a75a735/src/Microsoft.DotNet.XUnitExtensions/src/Discoverers/SkipOnCoreClrDiscoverer.cs#L50. However, in this case we expect it to perform stressMode || runtimeConfiguration, but I can see why we would need to keep && in other cases.

I have tested locally on macOS, it does not skip this test with RuntimeConfiguration=checked, and commenting /* , RuntimeTestModes.JitMinOpts */ part does skip it.

stephentoub commented 3 years ago

This was fixed.