dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x86: 1 Regression on 12/28/2022 6:20:56 PM #11264

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Burgers.Test3.html>) 153.65 ms 203.56 ms 1.32 0.16 False 2307366666.6666665 3110700000 1.3481602403894772 Trace Trace

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Burgers*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Burgers.Test3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 203.55764166666665 > 162.35867374999998. IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM. IsRegressionStdDev: Marked as regression because -46.06234216162504 (T) = (0 -207340425.7440476) / Math.Sqrt((13056736513586.77 / (29)) + (11162983392259.783 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.33003574762099974 = (155890866.92963856 - 207340425.7440476) / 155890866.92963856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```### Baseline Jit Disasm ```assembly ; BenchmarkDotNet.Autogenerated.Runnable_0.__ForDisassemblyDiagnoser__() push ebp mov ebp,esp push eax mov [ebp-4],ecx mov ecx,[ebp-4] cmp dword ptr [ecx+38],0B jne short M00_L00 mov ecx,[ebp-4] call dword ptr ds:[0A9A7630] mov esp,ebp pop ebp ret M00_L00: xor eax,eax mov esp,ebp pop ebp ret ; Total bytes of code 35 ``` ### Compare Jit Disasm ```assembly ; BenchmarkDotNet.Autogenerated.Runnable_0.__ForDisassemblyDiagnoser__() push ebp mov ebp,esp push eax mov [ebp-4],ecx mov ecx,[ebp-4] cmp dword ptr [ecx+38],0B jne short M00_L00 mov ecx,[ebp-4] call dword ptr ds:[0B207630] mov esp,ebp pop ebp ret M00_L00: xor eax,eax mov esp,ebp pop ebp ret ; Total bytes of code 35 ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
dakersnar commented 1 year ago

Dup of https://github.com/dotnet/runtime/issues/80129