dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 1 Regression on 6/29/2023 6:18:13 PM #19456

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline aef8d693286dbfdfc3c8460ea8a8699cf7c73eed
Compare 2f208128f257b2390a8edefa0a19e0eaad6bd419
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.Concurrent.AddRemoveFromDifferentThreads(Int32).ConcurrentStack(Size%3a%202000000).html>) 130.95 ms 162.15 ms 1.24 0.26 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>.ConcurrentStack(Size: 2000000) #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 162.15255 > 143.3993925. IsChangePoint: Marked as a change because one of 5/5/2023 9:43:39 PM, 6/22/2023 7:55:14 AM, 6/29/2023 1:56:51 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM. IsRegressionStdDev: Marked as regression because -7.653182943258631 (T) = (0 -168288936.67897123) / Math.Sqrt((158320166025812.8 / (31)) + (38139857451850.94 / (13))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (13) - 2, .025) and -0.14804697009244025 = (146587152.84568948 - 168288936.67897123) / 146587152.84568948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Compare](https://pvscmdupload.z22.web.core.windows.net/autofilereport/jitdasms/07_04_2023/System_Collections_Concurrent_AddRemoveFromDifferentThreads_Int32__ConcurrentStack_Size__2000000__compare_996cc849-6fd5-4a3f-b605-da97b8f92d5c.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
cincuranet commented 1 year ago

Looks like it's getting steadily slower and slower from ~Jun 20. Any ideas @EgorBo @AndyAyersMS?

EgorBo commented 1 year ago

Looks like static pgo