search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x86: 28 Improvements on 10/11/2023 8:44:35 PM #23205

Open performanceautofiler[bot] opened 12 months ago

performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
249.46 μs 218.29 μs 0.88 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.InstanceDelegate #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 218.285931712963 < 236.99197663926682. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/1/2023 6:36:20 PM, 8/8/2023 9:37:01 AM, 10/3/2023 2:46:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 15.166214213968017 (T) = (0 -218342.65074424335) / Math.Sqrt((238363083.53811473 / (299)) + (41959.29919263222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05850816287661162 = (231911.3582666444 - 218342.65074424335) / 231911.3582666444 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
223.70 μs 155.92 μs 0.70 0.05 False
225.17 μs 156.43 μs 0.69 0.04 False
234.20 μs 155.92 μs 0.67 0.05 False
228.16 μs 155.92 μs 0.68 0.05 False
225.17 μs 155.92 μs 0.69 0.06 False
227.42 μs 156.44 μs 0.69 0.04 False
223.66 μs 156.67 μs 0.70 0.03 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.CastingPerf.IFooFooIsIFoo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 155.92268659558263 < 211.78823222789114. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.488718507876857 (T) = (0 -156052.1876171366) / Math.Sqrt((928137270.0615565 / (299)) + (40601.85452236277 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19532218282758143 = (193931.2657648412 - 156052.1876171366) / 193931.2657648412 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.ObjScalarValueType #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.4337293729373 < 215.08215446428574. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.404381867104803 (T) = (0 -156155.6660442629) / Math.Sqrt((940374055.4355885 / (299)) + (67021.79278807151 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19567918905648 = (194145.99736774468 - 156155.6660442629) / 194145.99736774468 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.ObjInt #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 155.91804352310234 < 215.62146025276053. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.486799781240283 (T) = (0 -156008.33436824346) / Math.Sqrt((940212219.9851978 / (299)) + (47491.828087367896 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19638250681531536 = (194132.57637037296 - 156008.33436824346) / 194132.57637037296 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.ObjFooIsObj2 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 155.92108380480906 < 215.16444931382273. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.414373055283548 (T) = (0 -156102.47258832498) / Math.Sqrt((943172458.209471 / (299)) + (59367.63530580105 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19602527844265435 = (194163.40887676846 - 156102.47258832498) / 194163.40887676846 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.FooObjIsNull #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 155.92129664889563 < 213.4872825892857. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.393585368387352 (T) = (0 -156090.08173640116) / Math.Sqrt((954264273.71941 / (299)) + (65056.73279819297 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19681668296174085 = (194339.29767364167 - 156090.08173640116) / 194339.29767364167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.ObjObjrefValueType #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.44222321428572 < 215.60939595321824. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.385020706575915 (T) = (0 -156228.8708887108) / Math.Sqrt((939433672.6031215 / (299)) + (67175.62148401821 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1953848521333783 = (194165.9578531926 - 156228.8708887108) / 194165.9578531926 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.ObjFooIsObj #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.67020214521452 < 214.59422644927534. IsChangePoint: Marked as a change because one of 6/28/2023 3:57:02 AM, 7/14/2023 2:26:40 PM, 8/8/2023 9:37:01 AM, 9/1/2023 1:30:05 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.424403226475174 (T) = (0 -156148.6411975274) / Math.Sqrt((934953676.0177807 / (299)) + (91632.0468812771 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19542692763171884 = (194076.39474920524 - 156148.6411975274) / 194076.39474920524 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
742.58 μs 501.08 μs 0.67 0.01 False
3.74 ms 2.56 ms 0.68 0.01 False
185.82 μs 125.48 μs 0.68 0.04 False
1.86 ms 1.26 ms 0.67 0.01 False
387.25 μs 249.68 μs 0.64 0.01 False
944.95 μs 641.99 μs 0.68 0.01 False
2.79 ms 1.95 ms 0.70 0.01 False
2.22 ms 1.53 ms 0.69 0.01 False
553.35 μs 386.36 μs 0.70 0.01 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.GetMember.GetMethod4 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 501.0781451612904 < 704.7225587121212. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 115.5103354548237 (T) = (0 -506096.28448392294) / Math.Sqrt((186492535.67837423 / (299)) + (51293809.34668849 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3279049182743268 = (753012.9266598242 - 506096.28448392294) / 753012.9266598242 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod20 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.556347361111111 < 3.5392384979166667. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 197.0872330911336 (T) = (0 -2538292.7081278767) / Math.Sqrt((4200468421.776716 / (299)) + (325295239.3772389 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32675272306906905 = (3770223.5049489588 - 2538292.7081278767) / 3770223.5049489588 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod1 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 125.47758400537634 < 175.76714889705883. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 58.551013425505865 (T) = (0 -127286.53024349) / Math.Sqrt((14091848.014081923 / (299)) + (13406557.252538467 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3232662983154691 = (188089.53939584707 - 127286.53024349) / 188089.53939584707 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod10 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.255055354020979 < 1.7682240756766379. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 165.2802820430876 (T) = (0 -1268423.2784326128) / Math.Sqrt((1054938330.0276221 / (299)) + (132943424.37966345 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32580966519534377 = (1881402.3473061698 - 1268423.2784326128) / 1881402.3473061698 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod2 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 249.67617063492062 < 352.7078716085271. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 118.38992294492131 (T) = (0 -252501.90750204824) / Math.Sqrt((50110150.974369325 / (299)) + (12061122.321369613 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3291816671475826 = (376408.7758728556 - 252501.90750204824) / 376408.7758728556 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod5 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 641.98935 < 885.552081495098. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 199.80960843697673 (T) = (0 -634937.5822485207) / Math.Sqrt((249935427.4087555 / (299)) + (19544976.298830137 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32492658987421325 = (940545.9802219324 - 634937.5822485207) / 940545.9802219324 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod15 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9479145238095241 < 2.650608514880952. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 165.81317046499728 (T) = (0 -1896652.1177571537) / Math.Sqrt((2186192171.9882903 / (299)) + (311266591.59876305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32829867514660055 = (2823653.9777126433 - 1896652.1177571537) / 2823653.9777126433 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod12 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.5284356150793652 < 2.1214585736034803. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 168.25576801929012 (T) = (0 -1522492.710857519) / Math.Sqrt((1534259501.7143333 / (299)) + (181205105.05382523 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3255115432450163 = (2257255.399421288 - 1522492.710857519) / 2257255.399421288 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod3 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 386.3624801587302 < 527.7266648361515. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 114.9533484536487 (T) = (0 -380228.4227514339) / Math.Sqrt((96283832.25729467 / (299)) + (29015783.10112081 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32576227188719487 = (563938.2177792026 - 380228.4227514339) / 563938.2177792026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.89 ns 0.53 ns 0.28 0.23 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_Vector3.LengthBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.5332623350397006 < 1.794878975161174. IsChangePoint: Marked as a change because one of 10/6/2023 6:14:38 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 3.492145192133754 (T) = (0 -0.5418616918740096) / Math.Sqrt((0.08041019748139185 / (299)) + (0.0005001812113585975 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10152351815427195 = (0.6030894551194823 - 0.5418616918740096) / 0.6030894551194823 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.13 μs 972.36 ns 0.86 0.04 False
753.59 ns 660.35 ns 0.88 0.06 False
500.60 ns 471.24 ns 0.94 0.08 False
525.20 ns 423.33 ns 0.81 0.04 False
587.37 ns 468.58 ns 0.80 0.04 False
679.33 ns 581.59 ns 0.86 0.06 False
677.73 ns 582.06 ns 0.86 0.06 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 972.3646012773329 < 1.0715294598771863. IsChangePoint: Marked as a change because one of 8/27/2023 2:42:29 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 12.347808195709126 (T) = (0 -974.8210983059696) / Math.Sqrt((5792.214401413429 / (299)) + (13.139886214664465 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.054093393729202736 = (1030.5680199752137 - 974.8210983059696) / 1030.5680199752137 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 660.3487288135592 < 715.9233104642885. IsChangePoint: Marked as a change because one of 7/5/2023 7:07:19 PM, 7/14/2023 2:26:40 PM, 8/28/2023 1:23:07 PM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 13.763598675567847 (T) = (0 -660.1752033919674) / Math.Sqrt((2474.5864845473366 / (299)) + (0.8654216478143283 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.056797982099722664 = (699.9298038628309 - 660.1752033919674) / 699.9298038628309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 471.2404713346437 < 477.1641445113489. IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 7/12/2023 7:15:00 AM, 7/27/2023 4:03:50 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 31.985191434658997 (T) = (0 -472.0598428165765) / Math.Sqrt((283.0559257399244 / (299)) + (1.0312892590295744 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0642241725007551 = (504.4582569290159 - 472.0598428165765) / 504.4582569290159 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 423.33294001450906 < 497.2533141784571. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 9/11/2023 8:56:48 PM, 10/5/2023 11:31:53 PM, 10/9/2023 6:44:06 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 17.229708287653477 (T) = (0 -410.24284560212266) / Math.Sqrt((1689.0572483238032 / (299)) + (110.20315897288462 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.1259473171942463 = (469.35711504851423 - 410.24284560212266) / 469.35711504851423 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 468.5750653397668 < 557.019824834307. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 9/11/2023 8:56:48 PM, 10/5/2023 11:31:53 PM, 10/9/2023 6:44:06 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 23.30861228122533 (T) = (0 -467.7642575771966) / Math.Sqrt((1902.3203049603285 / (299)) + (3.1458769329381164 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.11300584785451319 = (527.358896837995 - 467.7642575771966) / 527.358896837995 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 581.5919754649601 < 645.4255250095517. IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 7/5/2023 7:07:19 PM, 7/17/2023 6:26:12 PM, 7/21/2023 12:22:03 PM, 8/28/2023 1:23:07 PM, 10/4/2023 10:43:01 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 26.238671153738554 (T) = (0 -582.2277500687512) / Math.Sqrt((1592.9351927370196 / (299)) + (1.6202113780889842 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09520980561962976 = (643.4947611998379 - 582.2277500687512) / 643.4947611998379 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 582.0620530198076 < 643.8969757485087. IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 7/5/2023 7:07:19 PM, 7/17/2023 6:26:12 PM, 7/21/2023 12:22:03 PM, 8/10/2023 8:47:51 AM, 10/4/2023 10:43:01 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 31.116803502360252 (T) = (0 -582.182154967095) / Math.Sqrt((854.2574092089438 / (299)) + (0.14911267362756672 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08300984112847874 = (634.8837545689125 - 582.182154967095) / 634.8837545689125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.70 μs 17.95 μs 0.87 0.80 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<Int32>.Array_ComparerClass(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.952828 < 20.49830675. IsChangePoint: Marked as a change because one of 7/11/2023 8:33:01 AM, 8/29/2023 11:09:08 AM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 3.64633966482491 (T) = (0 -13880.944865867534) / Math.Sqrt((19105465.774126444 / (299)) + (4120312.6192376167 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13949692937082786 = (16131.197365418155 - 13880.944865867534) / 16131.197365418155 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
943.02 ns 710.29 ns 0.75 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_BitOperations.TrailingZeroCount_ulong #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 710.2934007179582 < 895.901206197238. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 8/8/2023 9:37:01 AM, 8/28/2023 1:23:07 PM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 8.268590262825565 (T) = (0 -710.1462057881783) / Math.Sqrt((8169.224102924371 / (299)) + (0.15062098467860072 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05738083228868977 = (753.3755201609375 - 710.1462057881783) / 753.3755201609375 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare 0a77fd44c350894f0ea84a46f5188b9073214d26
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.25 μs 1.91 μs 0.85 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateFor<Int32>.IList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9129267729623243 < 2.133993170280746. IsChangePoint: Marked as a change because one of 7/12/2023 9:48:15 PM, 8/27/2023 2:42:29 PM, 9/23/2023 5:20:44 PM, 10/5/2023 11:31:53 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 28.191848837868164 (T) = (0 -1923.4986512196838) / Math.Sqrt((18480.890302222964 / (299)) + (25.29415628104859 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1047668610695144 = (2148.600814216555 - 1923.4986512196838) / 2148.600814216555 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)