search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.95k stars 4.65k forks source link

Test failure: baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.cmd #106742

Closed v-wenyuxu closed 1 week ago

v-wenyuxu commented 3 weeks ago

Failed in: runtime-coreclr r2r 20240819.1

Failed tests:

R2R-CG2 linux arm Checked @ (Debian.12.Arm32.Open)Ubuntu.2004.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-arm32v7
    - baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.cmd

Error message:

 /root/helix/work/workitem/e/baseservices/exceptions/baseservices-exceptions/../stackoverflow/stackoverflowtester/stackoverflowtester.sh: line 322: -r:/root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/IL-CG2/*.dll: No such file or directory

Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/stackoverflow/stackoverflowtester/output.txt
Raw output:
BEGIN EXECUTION
in takeLock
/root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/IL-CG2/stackoverflowtester.dll
07:24:31
Response file: /root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.dll.rsp
/root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/IL-CG2/stackoverflowtester.dll
-o:/root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.dll
-r:/root/helix/work/correlation/System.*.dll
-r:/root/helix/work/correlation/Microsoft.*.dll
-r:/root/helix/work/correlation/xunit.*.dll
-r:/root/helix/work/correlation/mscorlib.dll
--verify-type-and-field-layout
--method-layout:random
--targetarch:arm
--targetos:linux
Running CrossGen2:  dotnet /root/helix/work/correlation/crossgen2/crossgen2.dll @/root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.dll.rsp  
Emitting R2R PE file: /root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.dll
Running R2RDump:  dotnet /root/helix/work/correlation/R2RDump/R2RDump.dll --header --sc --in /root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.dll --out /root/helix/work/workitem/e/baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.dll.r2rdump --val
07:24:33
in ReleaseLock
in takeLock
/root/helix/work/correlation/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true stackoverflowtester.dll ''
Running stackoverflow test(smallframe main)
"Stack overflow."
"Repeated 174545 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionC()"
"   at TestStackOverflow.Program.InfiniteRecursionB()"
"   at TestStackOverflow.Program.InfiniteRecursionA()"
"--------------------------------"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program.Main(System.String[])"
""
Running stackoverflow test(largeframe main)
"Stack overflow."
"Repeated 42 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.InfiniteRecursionC2()"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program.Main(System.String[])"
""
Running stackoverflow test(smallframe secondary)
"Stack overflow."
"Repeated 174628 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB()"
"   at TestStackOverflow.Program.InfiniteRecursionA()"
"   at TestStackOverflow.Program.InfiniteRecursionC()"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB()"
"   at TestStackOverflow.Program.InfiniteRecursionA()"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program+<>c__DisplayClass7_0.<SecondaryThreadsTest>b__0()"
"   at System.Threading.Thread.StartCallback()"
""
Running stackoverflow test(largeframe secondary)
"Stack overflow."
"Repeated 42 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.InfiniteRecursionC2()"
"----------------------

Stack trace:

   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor134|0_135(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
v-wenyuxu commented 2 weeks ago

Failed in: runtime-coreclr jitstressregs 20240825.1

Failed tests:

coreclr linux arm Checked jitstressregs4 @ (Debian.12.Arm32.Open)Ubuntu.2004.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-arm32v7
    - baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.cmd

Error message:


Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/stackoverflow/stackoverflowtester/output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true stackoverflowtester.dll ''
Running stackoverflow test(smallframe main)
"Stack overflow."
"Repeated 174544 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionC()"
"   at TestStackOverflow.Program.InfiniteRecursionB()"
"   at TestStackOverflow.Program.InfiniteRecursionA()"
"--------------------------------"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program.Main(System.String[])"
""
Running stackoverflow test(largeframe main)
"Stack overflow."
"Repeated 42 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.InfiniteRecursionC2()"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program.Main(System.String[])"
""
Running stackoverflow test(smallframe secondary)
"Stack overflow."
"Repeated 174628 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB()"
"   at TestStackOverflow.Program.InfiniteRecursionA()"
"   at TestStackOverflow.Program.InfiniteRecursionC()"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB()"
"   at TestStackOverflow.Program.InfiniteRecursionA()"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program+<>c__DisplayClass7_0.<SecondaryThreadsTest>b__0()"
"   at System.Threading.Thread.StartCallback()"
""
Running stackoverflow test(largeframe secondary)
"Stack overflow."
"Repeated 42 times:"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.InfiniteRecursionC2()"
"--------------------------------"
"   at TestStackOverflow.Program.InfiniteRecursionB2()"
"   at TestStackOverflow.Program.InfiniteRecursionA2()"
"   at TestStackOverflow.Program.Test(Boolean)"
"   at TestStackOverflow.Program+<>c__DisplayClass7_0.<SecondaryThreadsTest>b__0()"
"   at System.Threading.Thread.StartCallback()"
""
Running stackoverflow3 test()
Call number 50 to the Execute method
Call number 100 to the Execute method
Call number 150 to the Execute method
Call number 200 to the Execute method
Call number 250 to the Execute method
Call number 300 to the Execute method
Call number 350 to the Execute method
Call number 400 to the Execute method
Call number 450 to the Execute method
Call number 500 to the Execute method
Call number 550 to the Execute method
Call number 600 to the Execute method
Call number 650 to the Execute method
Call number 700 to the Execute method
Call number 750 to the Execute method
Call number 800 to the Execute method
Call number 850 to the Execute method
Call number 900 to the Execute method
Call number 950 to the Execute method
Call number 1000 to the Execute method
"Stack overflow."
""
System.Exception: Missing "Main" method frame at the last line
   at TestStackOverflow.Program.TestStackOverflow3()
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /home/helixbot/dotnetbuild/dumps since 8/25/2024 11:48:15 AM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> /root/helix/work/workitem/e/baseservices/exceptions/baseservices-exceptions/../stackoverflow/stackoverflowtester/stackoverflowtester.sh

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor134|0_135(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
mikelle-rogers commented 2 weeks ago

@janvorli, are you looking into this failure?

janvorli commented 2 weeks ago

@mikelle-rogers yes, I am just about to start.

janvorli commented 2 weeks ago

The problem is caused by the fact that the libunwind is sometimes unable to unwind when the stack overflow occurs in a first instruction of a native function that pushes some registers to the stack. In some cases the unwind just fails, in some cases it unwinds incorrectly and in one of following unw_step calls it crashes in the libunwind due to an attempt to access invalid memory location. Since it reproduces relatively infrequently, I was unable to figure out the exact culprit.

JulieLeeMSFT commented 2 weeks ago

@janvorli can we disable this for now until we release .net9? We want to make sure our optional pipelines are clean. We can reenable later this year.

janvorli commented 2 weeks ago

I have investigated it further. The problem occurs when stack overflow occurs in a leaf function in a native shared library (ours or 3rd party). Those functions don't have unwind info at all on arm32, so the libunwind cannot unwind from that frame and we are unable to unwind to the first managed frame. I think we may be able to stop trying to unwind to the first managed frame and use the explicit frames for it, but for now, I'd disable the test on arm32 as you've suggested.