dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.14k stars 4.71k forks source link

superpmi: crossgen and PMI collection "clean phase" replay failures #47546

Open BruceForstall opened 3 years ago

BruceForstall commented 3 years ago

Related: https://github.com/dotnet/runtime/issues/47540

Part of the SuperPMI collection process is to "clean" the collection. This takes the collected assets, runs a SuperPMI replay on them, and delete the method contexts (functions) that fail to replay. Thus, the final result replay is error-free. As long as there aren't too many functions removed, we still get great coverage and utility from the collection.

However, each of these failures indicates a potential bug (or limitation) in SuperPMI. We should track down the failures, figure out why they occur, and fix SuperPMI or the JIT, VM, or JIT interface protocols, as necessary.

For a sample crossgen collection of the libraries using this command (Windows x64 Checked):

py c:\gh\runtime2\src\coreclr\scripts\superpmi.py collect --crossgen -output_mch_path c:\bugs\spmicollect4\win-x64-cg.mch -assemblies c:\gh\runtime2\artifacts\tests\coreclr\windows.x64.Checked\Tests\Core_Root

The "clean" phase encountered 680 failures (out of 197906 compilations).

category:eng-sys theme:super-pmi skill-level:expert cost:large

impact:medium

jakobbotsch commented 1 year ago

89443 fixed a bunch of cases but a lot still remain. 261 out of 1827 non-failing logs in https://dev.azure.com/dnceng/internal/_build/results?buildId=2230437&view=results still have either failures or misses:

jakobbotsch commented 1 year ago

After #89626 and #89654 we have 125 out of 1872 console logs with either failures or misses.

jakobbotsch commented 1 year ago

In the latest SPMI collection, which includes #90016 and #89852 we have 132/1882 logs with failures or misses: