dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.25k stars 4.45k forks source link

[mono][android] Investigate failures on Linux Bionic [x64/arm64] jobs #100281

Open matouskozak opened 1 month ago

matouskozak commented 1 month ago

Description

Linux bionic jobs have been lately unreliable and unusable due to extensive number of failures. These failures must be triaged and eliminated to gain reliable coverage on this platform. This is a part of improving the reliability and coverage of Mobile testing infrastructure (https://github.com/dotnet/runtime/issues/99949).

Error types

Overall, x64 Linux bionic is slightly more reliable than arm64 version. There are 3 main error patterns occuring on linux_bionic-[x64/arm64] Release AllSubsets_Mono jobs across many test suits.

  1. Segmentation faults - (exit code 139)
    dbug: Exit code: 139
    Std out:
    Std err:   Segmentation fault 
  2. Device not found - (exit code 81)
    XHarness exit code: 81 (DEVICE_NOT_FOUND)
  3. Operation got canceled (time out) - (exit code -4)
    • Test finishes without failures but the entire job gets cancelled (happening mostly on x64)
      ##[error]The Operation will be canceled. The next steps may not contain expected logs.
      ##[error]The operation was canceled.

Investigation areas

Goals


FYI: @kotlarmilos @vitek-karas

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
simonrozsival commented 1 month ago

/cc @rzikm