[android][infra] Flaky x64 Android failures in PayloadGroup0, failed to install mobile app

[matouskozak]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=676930 Build error leg or test failing: PayloadGroup0 Failing jobs: android_x64_release_allsubsets_mono_runtimetests, android_x64_release_allsubsets_mono_runtimetests_interp

Comment

A similar failure has been reported in https://github.com/dotnet/runtime/issues/83422, however the failing test group is different. Note, this failure doesn't seem to happen on every run.

GC/Features/HeapExpansion/bestfit-finalize/bestfit-finalize.sh [FAIL]
      System.AggregateException : One or more errors occurred. (Failed to install mobile app.) (The following constructor parameters did not have matching fixture data: _Global globalVar)
      ---- Failed to install mobile app.
      ---- The following constructor parameters did not have matching fixture data: _Global globalVar
      Stack Trace:

        ----- Inner Stack Trace #1 (Xunit.Sdk.TrueException) -----
           at GC_Features._Global..ctor()

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
        ----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

    GC/Features/HeapExpansion/bestfit-threaded/bestfit-threaded.sh [FAIL]
      System.AggregateException : One or more errors occurred. (Failed to install mobile app.) (The following constructor parameters did not have matching fixture data: _Global globalVar)
      ---- Failed to install mobile app.
      ---- The following constructor parameters did not have matching fixture data: _Global globalVar
      Stack Trace:

        ----- Inner Stack Trace #1 (Xunit.Sdk.TrueException) -----
           at GC_Features._Global..ctor()

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
        ----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

    GC/Features/HeapExpansion/bestfit/bestfit.sh [FAIL]
      System.AggregateException : One or more errors occurred. (Failed to install mobile app.) (The following constructor parameters did not have matching fixture data: _Global globalVar)
      ---- Failed to install mobile app.
      ---- The following constructor parameters did not have matching fixture data: _Global globalVar
      Stack Trace:

        ----- Inner Stack Trace #1 (Xunit.Sdk.TrueException) -----
           at GC_Features._Global..ctor()

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
        ----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

    GC/Features/HeapExpansion/bestfit_1/bestfit_1.sh [FAIL]
      System.AggregateException : One or more errors occurred. (Failed to install mobile app.) (The following constructor parameters did not have matching fixture data: _Global globalVar)
      ---- Failed to install mobile app.
      ---- The following constructor parameters did not have matching fixture data: _Global globalVar
      Stack Trace:

        ----- Inner Stack Trace #1 (Xunit.Sdk.TrueException) -----
           at GC_Features._Global..ctor()

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
        ----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----
...

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.AggregateException : One or more errors occurred. (Failed to install mobile app.) (The following constructor parameters did not have matching fixture data: _Global globalVar)",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=676930 Error message validated: [System.AggregateException : One or more errors occurred. (Failed to install mobile app.) (The following constructor parameters did not have matching fixture data: _Global globalVar)] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 5/24/2024 7:30:19 AM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[dotnet-policy-service[bot]]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @directhex, @matouskozak See info in area-owners.md if you want to be subscribed.

[akoeplinger]

FYI the actual underlying issue is a timeout while installing the .apk. From GC_Features_install.output.txt:

[9.0.0-prerelease.24256.1+62a8662088425e03f7ba44a4ec817c198e38d0f9] XHarness command issued: android install --package-name=net.dot.GC_Features --app=/datadisks/disk1/work/B59C0A37/w/B4DE097A/e/GC/Features/GC_Features.apk --output-directory=/datadisks/disk1/work/B59C0A37/w/B4DE097A/uploads/Reports/install --timeout 00:02:30
[22:14:46] info: Will attempt to find device supporting architectures: 'x86_64'
[22:14:47] info: Finding attached devices/emulators...
[22:14:47] info: Active Android device set to serial 'emulator-5556'
[22:14:48] info: Waiting for device to be available (max 5 minutes)

cmdLine:dotnet exec /datadisks/disk1/work/B59C0A37/p/microsoft.dotnet.xharness.cli/9.0.0-prerelease.24256.1/tools/net8.0/any/Microsoft.DotNet.XHarness.CLI.dll android install --package-name=net.dot.GC_Features --app=/datadisks/disk1/work/B59C0A37/w/B4DE097A/e/GC/Features/GC_Features.apk --output-directory=/datadisks/disk1/work/B59C0A37/w/B4DE097A/uploads/Reports/install --timeout 00:02:30 Timed Out (timeout in milliseconds: 240000, start: 5/15/2024 10:14:45 PM, end: 5/15/2024 10:18:45 PM)
xharness exitcode is : -100

[matouskozak]

Could it be caused by overloaded infrastructure? Only some builds are failing, e.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=686381&view=logs&j=353cbaf6-57aa-5b05-dc8e-ce12fdf7e230&t=4b37575b-d1a6-5cac-86e9-c8a12481ccef ran correctly.

[akoeplinger]

Not really, this is the emulator queue so once you get a Helix machine it should have emulators running on it already.

Most likely either a flaky issue or the one where emulators didn't start fast enough: https://github.com/dotnet/dnceng/issues/1448

[matouskozak]

Not really, this is the emulator queue so once you get a Helix machine it should have emulators running on it already.

Most likely either a flaky issue or the one where emulators didn't start fast enough: dotnet/dnceng#1448

Makes sense, thank you for the insight. cc: @ivanpovazan as possibly XHarness related