microsoft / mu_basecore

Project Mu BaseCore
https://microsoft.github.io/mu/
Other
239 stars 122 forks source link

[Cherry-Pick] BaseTools/HostBasedUnitTestRunner: Promote Unittest error to CI fail. #1039

Closed apop5 closed 1 month ago

apop5 commented 1 month ago

Description

Some unit tests would fail to execute or execute and not produce any output logs. In these cases, the only output would be in the CI Log as UnitTest Execution Error.

A UnitTest Execution Error should be considered the same as a unit tests test failing.

How This Was Tested

On repo where hosted based unit test failed execution prior to generating test results, CI would pass and CI Log would show "Execution Error" for the unit test.

After integrating this change, CI will fail with a unit test error.

Integration Instructions

For unit tests that are failing, each unit test will need to be examined and individually corrected.

Javagedes commented 1 month ago

supersedes https://github.com/microsoft/mu_basecore/pull/1035 and is a cherry-pick of https://github.com/tianocore/edk2/pull/5905

codecov-commenter commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 1.35%. Comparing base (e1eb585) to head (e23f7a7).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## release/202311 #1039 +/- ## =============================================== Coverage 1.35% 1.35% =============================================== Files 1304 1304 Lines 333854 333854 Branches 5103 5103 =============================================== Hits 4512 4512 Misses 329259 329259 Partials 83 83 ``` | [Flag](https://app.codecov.io/gh/microsoft/mu_basecore/pull/1039/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | Coverage Δ | | |---|---|---| | [MdeModulePkg](https://app.codecov.io/gh/microsoft/mu_basecore/pull/1039/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `0.68% <ø> (ø)` | | | [MdePkg](https://app.codecov.io/gh/microsoft/mu_basecore/pull/1039/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `5.40% <ø> (ø)` | | | [NetworkPkg](https://app.codecov.io/gh/microsoft/mu_basecore/pull/1039/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `0.55% <ø> (ø)` | | | [PolicyServicePkg](https://app.codecov.io/gh/microsoft/mu_basecore/pull/1039/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `30.41% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.