search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.27k stars 4.73k forks source link

superpmi-diffs pipeline improvements #61483

Open BruceForstall opened 2 years ago

BruceForstall commented 2 years ago

This issue tracks a number of improvement ideas for the runtime-coreclr superpmi-diffs AzDO pipeline.

category:eng-sys theme:super-pmi skill-level:intermediate cost:small impact:small

ghost commented 2 years ago

Tagging subscribers to this area: @JulieLeeMSFT See info in area-owners.md if you want to be subscribed.

Issue Details
This issue tracks a number of improvement ideas for the new `runtime-coreclr superpmi-asmdiffs` AzDO [pipeline](https://dev.azure.com/dnceng/public/_build?definitionId=1064&_a=summary). - [ ] JIT asserts (e.g., [this](https://github.com/dotnet/runtime/pull/61477) example) and other SuperPMI errors (such as https://github.com/dotnet/runtime/issues/61482) need to be surfaced as errors in the asmdiffs jobs (this is probably already true), and shown prominently in the "Extensions" page that summarizes the diffs. They should probably exist as a prominent section in each summary.md file as well as the overall summary.md files. - [ ] Currently, SuperPMI MISSING data causes superpmi.py to return a failure error code (of 1), which causes the AzDO asmdiffs job to be marked as "warning" (circle with exclamation point). We should change it to ignore these, at least for the AzDO job.
Author: BruceForstall
Assignees: -
Labels: `area-CodeGen-coreclr`
Milestone: 7.0.0
kunalspathak commented 2 years ago

We should consider uploading back subset of .dasm files as artifacts. It will be useful to have them if someone who sees the diffs and (most likely) want to investigate. Having them handy will avoid trouble of building and producing those dasm files. They can be zipped and uploaded. On my machine, I zipped a folder contined 42K .dasm files comprising of 612 MB, but after zipping they were just 30MB.

One option would be to pass a flag to jit-analyze which after producing the diff.md will delete all but top 20 .dasm files. Then, we can just zip that folder and upload as artifact.

kunalspathak commented 2 years ago

We should consider uploading back subset of .dasm files as artifacts.

This is done in https://github.com/dotnet/runtime/pull/61700.

EgorBo commented 2 years ago

https://dev.azure.com/dnceng/public/_build/results?buildId=1582186&view=results SPMI failed with NotFound in my jit pr https://github.com/dotnet/runtime/pull/64530

kunalspathak commented 2 years ago

https://dev.azure.com/dnceng/public/_build/results?buildId=1582186&view=results SPMI failed with NotFound in my jit pr #64530

It seems that win-x64 timed out. https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-64530-merge-dd476490704a459997/win-x64/1/console.af0c1c0e.log?sv=2019-07-07&se=2022-02-20T16%3A55%3A26Z&sr=c&sp=rl&sig=0lS82iJzZJaElO7kAHrUZaZsP9N9XszewTrYMIKMAF4%3D

EgorBo commented 2 years ago

https://dev.azure.com/dnceng/public/_build/results?buildId=1582186&view=results SPMI failed with NotFound in my jit pr #64530

It seems that win-x64 timed out. https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-64530-merge-dd476490704a459997/win-x64/1/console.af0c1c0e.log?sv=2019-07-07&se=2022-02-20T16%3A55%3A26Z&sr=c&sp=rl&sig=0lS82iJzZJaElO7kAHrUZaZsP9N9XszewTrYMIKMAF4%3D

ah, makes sense now, the diffs there were quite big

kunalspathak commented 2 years ago

the diffs there were quite big

Is that expected? I just want to make sure that we are picking up the correct baseline jit and do asmdiff only on your changes.

EgorBo commented 2 years ago

the diffs there were quite big

Is that expected? I just want to make sure that we are picking up the correct baseline jit and do asmdiff only on your changes.

Hm.. not sure, didn't expect it, running locally now

BruceForstall commented 1 year ago

Related: https://github.com/dotnet/runtime/issues/75848, https://github.com/dotnet/runtime/issues/80718, https://github.com/dotnet/runtime/issues/80719