search

dotnet / roslyn

The Roslyn .NET compiler provides C# and Visual Basic languages with rich code analysis APIs.
https://docs.microsoft.com/dotnet/csharp/roslyn-sdk/
MIT License
19.1k stars 4.04k forks source link

ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId} in Microsoft.Extensions.Logging.Generators.Roslyn4.0 #63863

Open carlossanlop opened 2 years ago

carlossanlop commented 2 years ago

Found in a release/7.0 backport PR: https://github.com/dotnet/runtime/pull/75284

Please help determine if this needs a fix to get backported to 7.0.

ghost commented 2 years ago

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

Issue Details
Found in a release/7.0 backport PR: https://github.com/dotnet/runtime/pull/75284 Please help determine if this needs a fix to get backported to 7.0. - Queue: `Build Browser wasm Linux Release _Threading_PerfTracing_BuildOnly` - Job results: https://dev.azure.com/dnceng-public/public/_build/results?buildId=8964&view=logs&j=8d661eb8-1694-5b60-6fbc-3f561a726c69&t=85fabc09-846c-57ca-7481-53449124685f&l=2817 - Build log message: ``` Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception: [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj] ##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj] ##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll ```
Author: carlossanlop
Assignees: -
Labels: `arch-wasm`, `area-Extensions-Logging`, `source-generator`
Milestone: -
ghost commented 2 years ago

Tagging subscribers to this area: @dotnet/area-extensions-logging See info in area-owners.md if you want to be subscribed.

Issue Details
Found in a release/7.0 backport PR: https://github.com/dotnet/runtime/pull/75284 Please help determine if this needs a fix to get backported to 7.0. - Queue: `Build Browser wasm Linux Release _Threading_PerfTracing_BuildOnly` - Job results: https://dev.azure.com/dnceng-public/public/_build/results?buildId=8964&view=logs&j=8d661eb8-1694-5b60-6fbc-3f561a726c69&t=85fabc09-846c-57ca-7481-53449124685f&l=2817 - Build log message: ``` Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception: [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj] ##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj] ##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll ```
Author: carlossanlop
Assignees: -
Labels: `arch-wasm`, `area-Extensions-Logging`, `source-generator`
Milestone: -
carlossanlop commented 2 years ago

@lewing @radical @steveisok @akoeplinger

radical commented 2 years ago

Looks similar to https://github.com/dotnet/runtime/issues/70836 .

steveisok commented 2 years ago

Should we run these tests on wasm/mobile at all?

radical commented 2 years ago

This is probably getting triggered because src/libraries has changes.

radical commented 2 years ago

This is not even running any tests, it's failing while building. And the will only build - Build Browser wasm Linux Release _Threading_PerfTracing_BuildOnly.

lewing commented 2 years ago

yeah this is a CSC crash it isn't really wasm related

tarekgh commented 2 years ago 
error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]

Does this suggest something not supported on WASM?

lewing commented 2 years ago

There is no wasm running at all at this point, it looks like the generator failed for some reason.

eerhardt commented 2 years ago

Looks like it is coming from this line in Roslyn:

https://github.com/dotnet/roslyn/blob/3e39dd3535962bf9e30bd650e4ff34b610b8349a/src/Compilers/Shared/BuildServerConnection.cs#L245

radical commented 2 years ago 
"/__w/1/s/.packages/microsoft.dotnet.arcade.sdk/7.0.0-beta.22418.4/tools/Build.proj" (default target) (1) ->
"/__w/1/s/Build.proj" (Build target) (3:2) ->
"/__w/1/s/src/libraries/oob.proj" (default target) (333) ->
"/__w/1/s/src/libraries/shims.proj" (default target) (334) ->
"/__w/1/s/src/libraries/shims/src/WindowsBase.csproj" (default target) (362) ->
"/__w/1/s/src/libraries/oob-src.proj" (default target) (339:23) ->
"/__w/1/s/src/libraries/Microsoft.Extensions.Caching.Memory/src/Microsoft.Extensions.Caching.Memory.csproj" (default target) (366:6) ->
"/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/src/Microsoft.Extensions.Logging.Abstractions.csproj" (default target) (389:7) ->
"/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj" (default target) (462:2) ->
(CoreGenerateSatelliteAssemblies target) ->
  /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception:  [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
  /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
tarekgh commented 2 years ago

Make sense @lewing.

Looking at the build analysis results too and I am seeing https://github.com/dotnet/runtime/pull/75284/checks?check_run_id=8258063425 all failures are on Mono only. This suggests specific to Mono.

ghost commented 2 years ago

Tagging subscribers to this area: @directhex See info in area-owners.md if you want to be subscribed.

Issue Details
Found in a release/7.0 backport PR: https://github.com/dotnet/runtime/pull/75284 Please help determine if this needs a fix to get backported to 7.0. - Queue: `Build Browser wasm Linux Release _Threading_PerfTracing_BuildOnly` - Job results: https://dev.azure.com/dnceng-public/public/_build/results?buildId=8964&view=logs&j=8d661eb8-1694-5b60-6fbc-3f561a726c69&t=85fabc09-846c-57ca-7481-53449124685f&l=2817 - Build log message: ``` Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception: [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj] ##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj] ##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll ```
Author: carlossanlop
Assignees: -
Labels: `arch-wasm`, `untriaged`, `area-Infrastructure-mono`, `source-generator`
Milestone: -
lewing commented 2 years ago

There is no mono running at this point either, perhaps it is a linker bug that is breaking the generator?

tarekgh commented 2 years ago

Ok, it looks make sense to move this to Roslyn repo then.

lewing commented 2 years ago

Actually looking at this as @eerhardt points out it appears to be the compiler service breaking

lewing commented 2 years ago

cc @MattGal 

        Unhandled exception: System.IO.IOException: Connection timed out : 'Global\msbuild-server-launch-NDSZDEInz8s3G_evUwEeKheECB+4j1X89xWLxyQAqoU'
           at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
           at Microsoft.Build.Experimental.MSBuildClient.TryLaunchServer()
MattGal commented 2 years ago

cc @MattGal

        Unhandled exception: System.IO.IOException: Connection timed out : 'Global\msbuild-server-launch-NDSZDEInz8s3G_evUwEeKheECB+4j1X89xWLxyQAqoU'
           at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
           at Microsoft.Build.Experimental.MSBuildClient.TryLaunchServer()

I noted yesterday on an email thread that we'd seen problems that led to finding out that "MSBuild Server by default" was a new behavior on Linux, but I'd ping someone like @rokonec for subject-matter-expertise as I only know about this in the context of investigating build failures.

If it happens to be related to the server running where it didn't previously and you can get a local repro, setting DOTNET_CLI_DO_NOT_USE_MSBUILD_SERVER = true and seeing it stop reproducing will confirm it. It's possible this is a red herring, of course.

akoeplinger commented 2 years ago

We actually have a quite long running tracking issue in runtime here: https://github.com/dotnet/runtime/issues/53420

According to runfo it fails about twice a week. Given how old the issue is and afaik we disabled the msbuild server in arcade recently I don't think it is related to that.