search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 59 Regressions on 10/27/2022 10:01:53 AM #9273

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f389dd342a331d97513d7600d6c302fae26e291a
Diff Diff

Regressions in SOD - Minimum Blazor Template - Publish - AOT

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/Microsoft.Extensions.Primitives.dll.br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 2.92 KB 2.88 KB 0.99 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/Microsoft.Extensions.Logging.Abstractions.dll.br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 8.75 KB 8.66 KB 0.99 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/blazor.boot.json.gz](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 1.67 KB 1.66 KB 0.99 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Aggregate - .dll](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 1.99 MB 2.04 MB 1.03 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Synthetic Wire Size - .gz](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 3.73 MB 3.78 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Synthetic Wire Size - .br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 2.72 MB 2.77 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - SOD - Minimum Blazor Template - Publish - AOT](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 17.94 MB 18.29 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/Microsoft.Extensions.Configuration.Abstractions.dll.br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 1.98 KB 2.01 KB 1.01 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/System.Private.CoreLib.dll.br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 359.59 KB 364.80 KB 1.01 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Aggregate - .br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 2.72 MB 2.77 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/dotnet.wasm.br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 1.92 MB 1.96 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/System.Private.CoreLib.dll.gz](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 427.48 KB 434.96 KB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/Microsoft.Extensions.Logging.dll.br](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 6.76 KB 6.84 KB 1.01 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Aggregate - .wasm](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 8.40 MB 8.59 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/System.Private.CoreLib.dll](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 1006.50 KB 1.03 MB 1.05 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 17.94 MB 18.29 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/dotnet.wasm.gz](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 2.77 MB 2.82 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Aggregate - .gz](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 3.73 MB 3.78 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - pub/wwwroot/_framework/dotnet.wasm](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 8.40 MB 8.59 MB 1.02 0.00 True
[SOD - Minimum Blazor Template - Publish - AOT - Total Uncompressed _framework](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=blazor_scenarios/SOD%20-%20Minimum%20Blazor%20Template%20-%20Publish%20-%20AOT.html>) 10.74 MB 10.99 MB 1.02 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'SOD - Minimum Blazor Template - Publish - AOT*'

Related Issues

Regressions

Improvements

### Payloads [Baseline](<>) [Compare](<>) ### Histogram ### Edge Detector Info [Collection Data]() #### SOD - Minimum Blazor Template - Publish - AOT ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsImprovementBase: Marked as improvement because the compare was 1% less than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsRegressionBase: Marked as regression because the compare was 1% greater than the baseline, and the value was not too small. IsImprovementBase: Marked as improvement because the compare was 1% less than the baseline, and the value was not too small. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 2 years ago

This might be result of API additions in https://github.com/dotnet/runtime/pull/76803. @stephentoub @lewing, do we care about that, ie. should we check if we can improve the size after trimming?

stephentoub commented 2 years ago

This might be result of API additions in https://github.com/dotnet/runtime/pull/76803.

Presumably we can diff a trimmed set of binaries before/after and see where the size difference is coming from?

cc: @eerhardt

eerhardt commented 2 years ago

@DrewScoggins @LoopedBard3 - it would be great if there were links added to these size regression issues to download the "before" and "after" apps, so they could be manually diffed.

eerhardt commented 2 years ago

Looking at the chart https://aka.ms/dotnetperfstatus for the SOD - Minimum Blazor Template - Publish - AOT for .NET 8, you can narrow the commit range down even further to https://github.com/dotnet/runtime/compare/2666e6c...46021ada, which includes https://github.com/dotnet/runtime/pull/76803. Its the bump on Oct 26 for the 2 selected points below:

image

However, you can also see 2 even worse regressions have occurred since then. The last one looks to be a major mistake where the app size doubled.

stephentoub commented 2 years ago

Did anyone follow-up on the doubling regression there?

eerhardt commented 2 years ago

I asked @SamMonoRT about the Blazor WASM size regressions in .NET 8. He said it was on his and @lewing's radar.

SamMonoRT commented 2 years ago

@stephentoub - the size regression causing double (or more) app size was due a reporting issue (https://github.com/dotnet/performance/issues/2749) That was fixed and newer graphs don't show that. However there are other smaller regressions in last couple weeks I'm investigating.

stephentoub commented 2 years ago

Ok, thanks.