On rare occasions, I sometimes see a test randomly fail due to the container crashing.
Here's an example of such a failure:
Failed Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyAppScenario(imageData: Version='3.1', VersionString='3.1', HasCustomSdk='False', SdkOS='bionic', Arch='Arm64', IsArm='True', OS='bionic', IsDistroless='False', Rid='linux-arm64') [24 s]
Error Message:
System.InvalidOperationException : Failed to execute docker build -t 3.1-build-132858653826941415 --target build --build-arg sdk_image=mcr.microsoft.com/dotnet/nightly/sdk:3.1-bionic-arm64v8 --build-arg runtime_image=mcr.microsoft.com/dotnet/nightly/runtime:3.1-bionic-arm64v8 --build-arg runtime_deps_image=mcr.microsoft.com/dotnet/nightly/runtime-deps:3.1-bionic-arm64v8 /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/bin/Debug/net6.0/consoleApp132858653757239058
Exit code: 134
Standard Error: The command '/bin/sh -c dotnet build --no-restore' returned a non-zero code: 134
Stack Trace:
at Microsoft.DotNet.Docker.Tests.DockerHelper.Execute(String args, Boolean ignoreErrors, Boolean autoRetry, ITestOutputHelper outputHelper) in /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/DockerHelper.cs:line 127
at Microsoft.DotNet.Docker.Tests.DockerHelper.ExecuteWithLogging(String args, Boolean ignoreErrors, Boolean autoRetry) in /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/DockerHelper.cs:line 142
at Microsoft.DotNet.Docker.Tests.DockerHelper.Build(String tag, String dockerfile, String target, String contextDir, Boolean pull, String[] buildArgs) in /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/DockerHelper.cs:line 50
at Microsoft.DotNet.Docker.Tests.ImageScenarioVerifier.BuildTestAppImage(String stageTarget, String contextDir, String[] customBuildArgs) in /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/ImageScenarioVerifier.cs:line 159
at Microsoft.DotNet.Docker.Tests.ImageScenarioVerifier.Execute() in /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/ImageScenarioVerifier.cs:line 55
at Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyAppScenario(ProductImageData imageData) in /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/RuntimeImageTests.cs:line 28
--- End of stack trace from previous location ---
Standard Output Messages:
Executing: docker run --name 3.1-create-console-132858653757239218 -w /app mcr.microsoft.com/dotnet/nightly/sdk:3.1-bionic-arm64v8 dotnet new console --framework netcoreapp3.1 --no-restore
Getting ready...
The template "Console Application" was created successfully.
Execution Elapsed Time: 00:00:06.0189880
Executing: docker cp 3.1-create-console-132858653757239218:/app /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/bin/Debug/net6.0/consoleApp132858653757239058
Execution Elapsed Time: 00:00:00.3620946
Executing: docker container rm -f 3.1-create-console-132858653757239218
3.1-create-console-132858653757239218
Execution Elapsed Time: 00:00:00.1730087
Executing: docker build -t 3.1-build-132858653826941415 --target build --build-arg sdk_image=mcr.microsoft.com/dotnet/nightly/sdk:3.1-bionic-arm64v8 --build-arg runtime_image=mcr.microsoft.com/dotnet/nightly/runtime:3.1-bionic-arm64v8 --build-arg runtime_deps_image=mcr.microsoft.com/dotnet/nightly/runtime-deps:3.1-bionic-arm64v8 /repo/dotnet-docker/tests/Microsoft.DotNet.Docker.Tests/bin/Debug/net6.0/consoleApp132858653757239058
Sending build context to Docker daemon 7.168kB
Step 1/11 : ARG sdk_image
Step 2/11 : ARG runtime_image
Step 3/11 : ARG runtime_deps_image
Step 4/11 : FROM $sdk_image as build
---> e623ec9d9504
Step 5/11 : EXPOSE 80
---> Using cache
---> 0967370806e0
Step 6/11 : WORKDIR app
---> Using cache
---> 4e093925e2b1
Step 7/11 : COPY NuGet.config .
---> Using cache
---> 7ad0b7e86846
Step 8/11 : COPY *.csproj .
---> 16c3f75c6188
Step 9/11 : RUN dotnet restore
---> Running in 8e56a4a7e6d8
Determining projects to restore...
Restored /app/app.csproj (in 517 ms).
Removing intermediate container 8e56a4a7e6d8
---> d1c4de8f2edc
Step 10/11 : COPY . .
---> fccf0396e101
Step 11/11 : RUN dotnet build --no-restore
---> Running in c6c7f24a1840
Microsoft (R) Build Engine version 16.7.2+b60ddb6f4 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.
app -> /app/bin/Debug/netcoreapp3.1/app.dll
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:04.79
\x1b[91mmalloc_consolidate(): invalid chunk size
\x1b[0m\x1b[91mAborted (core dumped)
\x1b[0m
The command '/bin/sh -c dotnet build --no-restore' returned a non-zero code: 134
These types of failures are nearly impossible to investigate today because they rarely happen, there is no known repro, and no crash dump is being collected by the test.
I propose that the tests be configured to automatically collect the core dump from the container and publish it as an artifact in the pipeline so that it can be investigated.
On rare occasions, I sometimes see a test randomly fail due to the container crashing.
Here's an example of such a failure:
These types of failures are nearly impossible to investigate today because they rarely happen, there is no known repro, and no crash dump is being collected by the test.
I propose that the tests be configured to automatically collect the core dump from the container and publish it as an artifact in the pipeline so that it can be investigated.