dotnet / fsharp

The F# compiler, F# core library, F# language service, and F# tooling integration for Visual Studio
https://dotnet.microsoft.com/languages/fsharp
MIT License
3.89k stars 783 forks source link

Timeout of MacOS CI leg #16878

Closed majocha closed 7 months ago

majocha commented 7 months ago

MacOS CI run occasionally times out after 60 minutes.

Sometimes it looks like it didn't just run out of time, but got stuck in a specific moment, like this:

Test run for /Users/runner/work/1/s/artifacts/bin/FSharp.Compiler.UnitTests/Release/net8.0/FSharp.Compiler.UnitTests.dll (.NETCoreApp,Version=v8.0)
Microsoft (R) Test Execution Command Line Tool Version 17.8.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
---------------------------------------------------------------
Executables

CSC                      = /Users/runner/work/1/s//.packages/Microsoft.Net.Compilers/4.3.0-1.22220.8/tools/csc.exe
BUILD_CONFIG             = Release
csc_flags                = /nologo
FSC                      = /Users/runner/work/1/s/tests/FSharp.Test.Utilities/../../artifacts/bin/fsc/Release/net8.0/fsc.dll
fsc_flags                = -r:System.Core.dll --nowarn:20 --define:COMPILED --preferreduilang:en-US
FSCOREDLLPATH            = /Users/runner/work/1/s/tests/FSharp.Test.Utilities/../../artifacts/bin/FSharp.Core/Release/netstandard2.0/FSharp.Core.dll
FSI                      = /Users/runner/work/1/s/tests/FSharp.Test.Utilities/../../artifacts/bin/fsi/Release/net8.0/fsi.dll
DotNetExe                =dotnet
DOTNET_MULTILEVEL_LOOKUP = 0
DOTNET_ROOT              = /Users/runner/.dotnet
FSI_FOR_SCRIPTS          = /Users/runner/work/1/s/tests/FSharp.Test.Utilities/../../artifacts/bin/fsi/Release/net8.0/fsi.dll
fsi_flags                = -r:System.Core.dll --nowarn:20 --define:INTERACTIVE --maxerrors:1 --abortonerror --preferreduilang:en-US
ILDASM                   = /Users/runner/work/1/s//.packages/runtime.osx-x64.Microsoft.NETCore.ILDAsm/5.0.0-preview.7.20364.11/runtimes/osx-x64/native/ildasm
PEVERIFY                 = dummy
---------------------------------------------------------------
[xUnit.net 00:00:03.01]     FSharp.Compiler.UnitTests.AssemblySigning.AssemblyKeyNameAttribute DESKTOP [SKIP]
  Skipped FSharp.Compiler.UnitTests.AssemblySigning.AssemblyKeyNameAttribute DESKTOP [1 ms]
[xUnit.net 00:00:10.58]     FSharp.Compiler.UnitTests.AssemblySigning.--keycontainer:name DESKTOP [SKIP]
  Skipped FSharp.Compiler.UnitTests.AssemblySigning.--keycontainer:name DESKTOP [1 ms]
value: "<null>"
value: "<null>"
##[error]The Operation will be canceled. The next steps may not contain expected logs.
##[error]The operation was canceled.
Finishing: Build / Test

See: https://dev.azure.com/dnceng-public/public/_build/results?buildId=603388&view=logs&j=7bab896a-24f8-544f-51eb-43745367a332&t=999dbed9-85e3-59ab-57f0-3e22828b5bad

The PR (#16877) doesn't touch any code, so most probably is not to blame here.

Maybe it's just a coincidence, or I'm not reading this right, but I've seen timeouts with logs like this in my draft here: (#16779) and they were related to a a deadlock and not just a simple timeout.

OK., you can toggle timestamps in the log above: image

9 minutes apparently doing nothing before it times out.

Compare with normal MacOS timeout (happens quite often too, unfortunately): image

vzarytovskii commented 7 months ago

Mac VMs are kinda limited and unpredictable, so it's kinda expected I should say.

majocha commented 7 months ago

Half an hour of lag in Linux tests looks sus, too: https://dev.azure.com/dnceng-public/public/_build/results?buildId=606491&view=logs&j=22fea640-1099-5f32-ec5d-316ad83f359a&t=4607427e-2085-572d-4b25-975ef0413f8f

image

vzarytovskii commented 7 months ago

Yeah, it's just infrastructure having hiccups, nothing we can do about it really.

abonie commented 7 months ago

It is not really actionable for us, if this starts happening on a regular basis we will revisit this issue