search

dotnet / aspire

Tools, templates, and packages to accelerate building observable, production-ready apps
https://learn.microsoft.com/dotnet/aspire
MIT License
3.95k stars 485 forks source link

Random test failure on windows CI - `Aspire.Azure.Storage.Queues.Tests.ConformanceTests.TracingEnablesTheRightActivitySource` #2105

Open radical opened 9 months ago

radical commented 9 months ago

Aspire.Azure.Storage.Queues.Tests.ConformanceTests.TracingEnablesTheRightActivitySource

Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Half-way through waiting for remote process.
Memory load: 32

Image Name                     PID Services

========================= ======== ============================================
svchost.exe                    972 BrokerInfrastructure, DcomLaunch, PlugPlay,
Power, SystemEventsBroker

svchost.exe                    588 RpcEptMapper, RpcSs

svchost.exe                    776 LSM

svchost.exe                   1224 TermService

svchost.exe                   1220 DsmSvc

svchost.exe                   1296 CoreMessagingRegistrar

svchost.exe                   1304 lmhosts

svchost.exe                   1348 EventLog

svchost.exe                   1392 NcbService

svchost.exe                   1400 TimeBrokerSvc

svchost.exe                   1516 nsi

svchost.exe                   1584 Dhcp

svchost.exe                   1576 FontCache

svchost.exe                   1592 DispBrokerDesktopSvc

svchost.exe                   1600 gpsvc

svchost.exe                   1668 UmRdpService

svchost.exe                   1676 vmicheartbeat

svchost.exe                   1696 vmickvpexchange

svchost.exe                   1756 vmicshutdown

svchost.exe                   1804 vmictimesync

svchost.exe                   1916 NlaSvc

svchost.exe                   1956 Schedule

svchost.exe                   2092 ProfSvc

svchost.exe                   2116 EventSystem

svchost.exe                   2128 Themes

svchost.exe                   2188 CertPropSvc

svchost.exe                   2228 netprofm

svchost.exe                   2260 SENS

svchost.exe                   2344 Dnscache

svchost.exe                   2340 Wcmsvc

svchost.exe                   2468 LanmanWorkstation

svchost.exe                   2496 WinHttpAutoProxySvc

svchost.exe                   2508 UserManager

svchost.exe                   2612 ShellHWDetection

svchost.exe                   2700 BFE, mpssvc

svchost.exe                   2808 SessionEnv

svchost.exe                   3004 iphlpsvc

svchost.exe                   3020 DiagTrack

svchost.exe                   3040 CryptSvc

svchost.exe                   3060 RemoteRegistry

svchost.exe                   2712 sacsvr

svchost.exe                   3076 StateRepository

svchost.exe                   3104 LanmanServer

svchost.exe                   3112 SysMain

svchost.exe                   3180 TrkWks
Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 225
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 55
   at Aspire.Azure.Storage.Queues.Tests.ConformanceTests.TracingEnablesTheRightActivitySource() in /_/tests/Aspire.Azure.Storage.Queues.Tests/ConformanceTests.cs:line 116
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Build.

This has failed 4 times in the last two weeks.

cc @joperezr @eerhardt

danmoseley commented 7 months ago

seems to be happening more now eg https://dev.azure.com/dnceng-public/public/_build/results?buildId=636719&view=ms.vss-test-web.build-test-results-tab&runId=15677388&resultId=100393&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

[testcontainers.org 00:00:00.89] Docker container efd3db82eb8a created
[testcontainers.org 00:00:00.95] Start Docker container efd3db82eb8a
[testcontainers.org 00:00:01.53] Wait for Docker container efd3db82eb8a to complete readiness checks
[testcontainers.org 00:00:01.54] Docker container efd3db82eb8a ready
[testcontainers.org 00:00:01.73] Docker container 2370c2be5f42 created
  Passed Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.ConformanceTests.ServiceLifetimeIsAsExpected(useKey: False) [3 ms]
  Passed Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.ConformanceTests.DbContextCanBeAlwaysResolved [< 1 ms]
  Passed Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.ConformanceTests.ConfigurationSchemaInvalidJsonConfigTest [4 ms]
  Passed Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.ConformanceTests.DbContextPoolingRegistersIDbContextPool [< 1 ms]
[testcontainers.org 00:00:01.73] Start Docker container 2370c2be5f42
[testcontainers.org 00:00:02.30] Wait for Docker container 2370c2be5f42 to complete readiness checks
[testcontainers.org 00:00:02.31] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:11.39] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:20.46] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:29.53] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:38.60] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:47.68] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:55.27] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:56.37] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:57.45] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:00:58.52] Execute "/opt/mssql-tools/bin/sqlcmd -Q SELECT 1;" at Docker container 2370c2be5f42
[testcontainers.org 00:02:05.77] Delete Docker container 5be70e7236c6
[xUnit.net 00:02:09.63]     Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.EnrichSqlServerTests.TracingEnablesTheRightActivitySource [FAIL]
[xUnit.net 00:02:09.64]       Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Half-way through waiting for remote process.
[xUnit.net 00:02:09.64]       Timed out at 4/9/2024 7:31:53 PM after 60000ms waiting for remote process.
[xUnit.net 00:02:09.64]         Process ID: 23772
[xUnit.net 00:02:09.64]         Handle: 4980
[xUnit.net 00:02:09.64]         Name: dotnet
[xUnit.net 00:02:09.64]         MainModule: /datadisks/disk1/work/AC8809AE/p/dotnet-cli/dotnet
[xUnit.net 00:02:09.64]         StartTime: 4/9/2024 7:30:53 PM
[xUnit.net 00:02:09.64]         TotalProcessorTime: 00:00:02.1800000
[xUnit.net 00:02:09.64]       
[xUnit.net 00:02:09.64]       Stack Trace:
[xUnit.net 00:02:09.64]         /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs(225,0): at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing)
[xUnit.net 00:02:09.64]         /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs(55,0): at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose()
[xUnit.net 00:02:09.64]         /_/tests/Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests/ConformanceTests.cs(116,0): at Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.ConformanceTests.TracingEnablesTheRightActivitySource()
[xUnit.net 00:02:09.64]            at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[xUnit.net 00:02:09.64]            at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
[xUnit.net 00:02:10.39]   Finished:    Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests
Data collector 'Blame' message: All tests finished running, Sequence file will not be generated.
  Failed Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.EnrichSqlServerTests.TracingEnablesTheRightActivitySource [1 m]
  Error Message:
   Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Half-way through waiting for remote process.
Timed out at 4/9/2024 7:31:53 PM after 60000ms waiting for remote process.
    Process ID: 23772
    Handle: 4980
    Name: dotnet
    MainModule: /datadisks/disk1/work/AC8809AE/p/dotnet-cli/dotnet
    StartTime: 4/9/2024 7:30:53 PM
    TotalProcessorTime: 00:00:02.1800000

  Stack Trace:
     at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 225
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 55
   at Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests.ConformanceTests.TracingEnablesTheRightActivitySource() in /_/tests/Aspire.Microsoft.EntityFrameworkCore.SqlServer.Tests/ConformanceTests.cs:line 116
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Results File: /datadisks/disk1/work/AC8809AE/w/BFB40A69/uploads/logs/TestResults.trx

Test Run Failed.
Total tests: 101
     Passed: 74
     Failed: 1
eerhardt commented 4 months ago

I haven't seen this in a while. Would be good to confirm if this still occurs.