search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.47k stars 4.77k forks source link

Assertion failed '!"Missing flags on tree"' #95042

Closed BruceForstall closed 1 year ago

BruceForstall commented 1 year ago

runtime-coreclr libraries-pgo net9.0-windows-Release-x64-CoreCLR_checked-jitosr_stress_random-Windows.10.Amd64.Open System.Configuration.ConfigurationManager.Tests

https://dev.azure.com/dnceng-public/public/_build/results?buildId=475007&view=ms.vss-test-web.build-test-results-tab&runId=10851678&paneView=debug&resultId=203727

C:\h\w\B80F097F\w\9E5F08FF\e>set DOTNET 
DOTNET_JitRandomOnStackReplacement=15
DOTNET_OSR_HitLimit=2
DOTNET_TC_OnStackReplacement=1
DOTNET_TC_OnStackReplacement_InitialCounter=1
DOTNET_TC_QuickJitForLoops=1
DOTNET_TieredCompilation=1

C:\h\w\B80F097F\w\9E5F08FF\e>call RunTests.cmd --runtime-path C:\h\w\B80F097F\p 
----- start Sun 11/19/2023 17:12:04.78 ===============  To repro directly: =====================================================
pushd C:\h\w\B80F097F\w\9E5F08FF\e\
"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================

C:\h\w\B80F097F\w\9E5F08FF\e>"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Configuration.ConfigurationManager.Tests (found 512 of 515 test cases)
  Starting:    System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 4)
Missing flags on tree [000189]: --X-------
               [000189] --C-G------                         *  CALL      void   System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this
               [000145] ----------- this                    +--*  LCL_VAR   ref    V18 tmp11        
               [000166] ----------- arg1                    \--*  LCL_VAR   ref    V19 tmp12        

Assert failure(PID 10928 [0x00002ab0], Thread: 10932 [0x2ab4]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR)

    File: D:\a\_work\1\s\src\coreclr\jit\fgdiagnostic.cpp Line: 3399
    Image: C:\h\w\B80F097F\p\dotnet.exe

----- end Sun 11/19/2023 17:12:24.03 ----- exit code -1073740286 ----------------------------------------------------------

Also fails linux-arm64:

+ grep DOTNET
DOTNET_TC_QuickJitForLoops=1
DOTNET_EnableCrashReport=1
DOTNET_TieredCompilation=1
DOTNET_JitRandomOnStackReplacement=15
DOTNET_TC_OnStackReplacement=1
DOTNET_OSR_HitLimit=2
DOTNET_DbgMiniDumpName=/home/helixbot/dotnetbuild/dumps/coredump.%d.dmp
DOTNET_DbgEnableMiniDump=1
DOTNET_TC_OnStackReplacement_InitialCounter=1
+ ./RunTests.sh --runtime-path /root/helix/work/correlation
----- start Sun Nov 19 16:58:39 UTC 2023 =============== To repro directly: =====================================================
pushd .
/root/helix/work/correlation/dotnet exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Configuration.ConfigurationManager.Tests (found 510 of 515 test cases)
  Starting:    System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 2)
Missing flags on tree [000189]: --X-------
               [000189] --C-G------                         *  CALL      void   System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this
               [000145] ----------- this                    +--*  LCL_VAR   ref    V18 tmp11        
               [000166] ----------- arg1                    \--*  LCL_VAR   ref    V19 tmp12        

Assert failure(PID 28 [0x0000001c], Thread: 28 [0x001c]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR)

    File: /__w/1/s/src/coreclr/jit/fgdiagnostic.cpp Line: 3399
    Image: /root/helix/work/correlation/dotnet

[createdump] Gathering state for process 28 dotnet
[createdump] Crashing thread 001c signal 6 (0006)
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228)
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228)
[createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp
[createdump] Written 181579776 bytes (44331 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 421ms
waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
./RunTests.sh: line 204:    28 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Sun Nov 19 16:58:49 UTC 2023 ----- exit code 134 ----------------------------------------------------------

@AndyAyersMS @dotnet/jit-contrib

ghost commented 1 year ago

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch See info in area-owners.md if you want to be subscribed.

Issue Details
runtime-coreclr libraries-pgo net9.0-windows-Release-x64-CoreCLR_checked-jitosr_stress_random-Windows.10.Amd64.Open System.Configuration.ConfigurationManager.Tests https://dev.azure.com/dnceng-public/public/_build/results?buildId=475007&view=ms.vss-test-web.build-test-results-tab&runId=10851678&paneView=debug&resultId=203727 ``` C:\h\w\B80F097F\w\9E5F08FF\e>set DOTNET DOTNET_JitRandomOnStackReplacement=15 DOTNET_OSR_HitLimit=2 DOTNET_TC_OnStackReplacement=1 DOTNET_TC_OnStackReplacement_InitialCounter=1 DOTNET_TC_QuickJitForLoops=1 DOTNET_TieredCompilation=1 C:\h\w\B80F097F\w\9E5F08FF\e>call RunTests.cmd --runtime-path C:\h\w\B80F097F\p ----- start Sun 11/19/2023 17:12:04.78 =============== To repro directly: ===================================================== pushd C:\h\w\B80F097F\w\9E5F08FF\e\ "C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing popd =========================================================================================================== C:\h\w\B80F097F\w\9E5F08FF\e>"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None) Discovered: System.Configuration.ConfigurationManager.Tests (found 512 of 515 test cases) Starting: System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 4) Missing flags on tree [000189]: --X------- [000189] --C-G------ * CALL void System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this [000145] ----------- this +--* LCL_VAR ref V18 tmp11 [000166] ----------- arg1 \--* LCL_VAR ref V19 tmp12 Assert failure(PID 10928 [0x00002ab0], Thread: 10932 [0x2ab4]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR) File: D:\a\_work\1\s\src\coreclr\jit\fgdiagnostic.cpp Line: 3399 Image: C:\h\w\B80F097F\p\dotnet.exe ----- end Sun 11/19/2023 17:12:24.03 ----- exit code -1073740286 ---------------------------------------------------------- ``` Also fails linux-arm64: ``` + grep DOTNET DOTNET_TC_QuickJitForLoops=1 DOTNET_EnableCrashReport=1 DOTNET_TieredCompilation=1 DOTNET_JitRandomOnStackReplacement=15 DOTNET_TC_OnStackReplacement=1 DOTNET_OSR_HitLimit=2 DOTNET_DbgMiniDumpName=/home/helixbot/dotnetbuild/dumps/coredump.%d.dmp DOTNET_DbgEnableMiniDump=1 DOTNET_TC_OnStackReplacement_InitialCounter=1 + ./RunTests.sh --runtime-path /root/helix/work/correlation ----- start Sun Nov 19 16:58:39 UTC 2023 =============== To repro directly: ===================================================== pushd . /root/helix/work/correlation/dotnet exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing popd =========================================================================================================== /root/helix/work/workitem/e /root/helix/work/workitem/e Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None) Discovered: System.Configuration.ConfigurationManager.Tests (found 510 of 515 test cases) Starting: System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 2) Missing flags on tree [000189]: --X------- [000189] --C-G------ * CALL void System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this [000145] ----------- this +--* LCL_VAR ref V18 tmp11 [000166] ----------- arg1 \--* LCL_VAR ref V19 tmp12 Assert failure(PID 28 [0x0000001c], Thread: 28 [0x001c]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR) File: /__w/1/s/src/coreclr/jit/fgdiagnostic.cpp Line: 3399 Image: /root/helix/work/correlation/dotnet [createdump] Gathering state for process 28 dotnet [createdump] Crashing thread 001c signal 6 (0006) apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228) apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228) [createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp.crashreport.json [createdump] Crash report successfully written [createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp [createdump] Written 181579776 bytes (44331 pages) to core file [createdump] Target process is alive [createdump] Dump successfully written in 421ms waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0 ./RunTests.sh: line 204: 28 Aborted (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE /root/helix/work/workitem/e ----- end Sun Nov 19 16:58:49 UTC 2023 ----- exit code 134 ---------------------------------------------------------- ``` @AndyAyersMS @dotnet/jit-contrib
Author: BruceForstall
Assignees: -
Labels: `JitStress`, `area-CodeGen-coreclr`, `blocking-clean-ci-optional`
Milestone: 9.0.0
AndyAyersMS commented 1 year ago

So far I can't repro this, it also did not recur in last weekend's lab runs. Will keep trying.

AndyAyersMS commented 1 year ago

Got it to repro -- looks like the same issue that I fixed over in #94689 via 3882492 -- the fgEntryBB was getting lost.

AndyAyersMS commented 1 year ago

Fixed via https://github.com/dotnet/runtime/pull/94689