search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 21 Improvements on 10/11/2023 8:44:35 PM #23146

Open performanceautofiler[bot] opened 12 months ago

performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.88 ns 3.46 ns 0.71 0.25 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.4562582895481717 < 4.63634774924049. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/28/2023 2:34:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 9.809030878555934 (T) = (0 -3.456349993853966) / Math.Sqrt((0.5847877416531594 / (230)) + (2.4248902457725146E-05 / (15))) is greater than 1.9697743954265454 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (230) + (15) - 2, .975) and 0.1252217070579099 = (3.9511154103166284 - 3.456349993853966) / 3.9511154103166284 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.08 ns 7.99 ns 0.88 0.17 False
43.07 ns 22.33 ns 0.52 0.01 True
26.00 ns 13.14 ns 0.51 0.21 False
24.25 ns 18.70 ns 0.77 0.20 False
9.13 ns 8.00 ns 0.88 0.21 False
26.36 ns 13.28 ns 0.50 0.17 False
9.06 ns 7.95 ns 0.88 0.17 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.993366365543726 < 8.605630206749034. IsChangePoint: Marked as a change because one of 9/14/2023 11:50:12 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 16.364284826921814 (T) = (0 -7.903529252494158) / Math.Sqrt((0.41185761176490643 / (227)) + (0.005715000258920377 / (15))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (15) - 2, .975) and 0.088433400553746 = (8.670270781416614 - 7.903529252494158) / 8.670270781416614 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.325111373477828 < 40.96780696224795. IsChangePoint: Marked as a change because one of 10/14/2023 5:24:18 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 149.21278300405595 (T) = (0 -22.066498584102145) / Math.Sqrt((0.1374544842812727 / (240)) + (0.07758078101679966 / (4))) is greater than 1.96981513413532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (240) + (4) - 2, .975) and 0.48862664072081835 = (43.15144342913463 - 22.066498584102145) / 43.15144342913463 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.14430669652518 < 24.868294709711073. IsChangePoint: Marked as a change because one of 10/11/2023 2:37:28 PM, 10/14/2023 5:24:18 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 72.44505764082028 (T) = (0 -13.117457193447041) / Math.Sqrt((6.543496649287166 / (239)) + (0.030881584951436277 / (4))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (239) + (4) - 2, .975) and 0.508522622743439 = (26.68984942230508 - 13.117457193447041) / 26.68984942230508 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.70289059747054 < 23.116225203685058. IsChangePoint: Marked as a change because one of 8/27/2023 2:42:29 PM, 9/16/2023 6:40:29 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 20.246767223004063 (T) = (0 -18.908032335241227) / Math.Sqrt((5.53487268041559 / (227)) + (0.07905258663862902 / (15))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (15) - 2, .975) and 0.15568514075423057 = (22.394527501425078 - 18.908032335241227) / 22.394527501425078 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.997909414815271 < 8.61943626299519. IsChangePoint: Marked as a change because one of 9/12/2023 11:09:48 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 14.466708835159068 (T) = (0 -7.9223679977024215) / Math.Sqrt((0.43848059887144314 / (228)) + (0.014522313811933595 / (15))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (15) - 2, .975) and 0.08940984975008856 = (8.700256636345262 - 7.9223679977024215) / 8.700256636345262 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.284894653052312 < 25.14129941391304. IsChangePoint: Marked as a change because one of 10/11/2023 2:37:28 PM, 10/14/2023 5:24:18 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 71.43253475665443 (T) = (0 -13.091693721838473) / Math.Sqrt((5.555618241463558 / (239)) + (0.05578621491862611 / (4))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (239) + (4) - 2, .975) and 0.5127328047413495 = (26.867586919922154 - 13.091693721838473) / 26.867586919922154 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.947060908664097 < 8.601393115191007. IsChangePoint: Marked as a change because one of 9/18/2023 2:28:59 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 16.58909070697959 (T) = (0 -7.903266250434866) / Math.Sqrt((0.4179233731675531 / (229)) + (0.004331748330721847 / (15))) is greater than 1.96981513413532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (15) - 2, .975) and 0.08801065009933677 = (8.665963315575686 - 7.903266250434866) / 8.665963315575686 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
29.68 ns 25.44 ns 0.86 0.18 False
21.64 ns 15.41 ns 0.71 0.22 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 25.435743174590314 < 28.16655546990875. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 8:56:48 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 8.458530773630391 (T) = (0 -25.52981584046294) / Math.Sqrt((7.962339440903363 / (230)) + (0.010746228248276033 / (15))) is greater than 1.9697743954265454 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (230) + (15) - 2, .975) and 0.058628982606131544 = (27.119823500771002 - 25.52981584046294) / 27.119823500771002 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.408305794173161 < 20.53791243798457. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 22.44866389063841 (T) = (0 -15.47265140636489) / Math.Sqrt((7.369761230346588 / (228)) + (0.024700717354177136 / (15))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (15) - 2, .975) and 0.21098832372823245 = (19.610167848815816 - 15.47265140636489) / 19.610167848815816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
131.78 ΞΌs 109.57 ΞΌs 0.83 0.05 False
131.41 ΞΌs 109.44 ΞΌs 0.83 0.04 False
284.61 ΞΌs 263.47 ΞΌs 0.93 0.03 False
218.95 ΞΌs 197.28 ΞΌs 0.90 0.03 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.GenericGenericMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.57005281177156 < 124.77995257262037. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 9/28/2023 2:34:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 13.348553454598495 (T) = (0 -109549.0491972344) / Math.Sqrt((119337174.59015186 / (227)) + (15389.274880757006 / (15))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (15) - 2, .975) and 0.08124963561232429 = (119237.0130598491 - 109549.0491972344) / 119237.0130598491 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.ClassVirtualMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.43751025416891 < 124.82513758967528. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/28/2023 2:34:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 13.623702772339955 (T) = (0 -109581.29268703518) / Math.Sqrt((119903071.40247671 / (228)) + (155300.0397868496 / (15))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (15) - 2, .975) and 0.08344467872150182 = (119557.75079040599 - 109581.29268703518) / 119557.75079040599 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 263.4724235576923 < 270.4109019694368. IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/28/2023 1:23:07 PM, 9/15/2023 8:45:09 AM, 9/19/2023 8:08:41 AM, 9/28/2023 2:34:07 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 12.601303874805689 (T) = (0 -262945.45953813544) / Math.Sqrt((293410839.52476674 / (221)) + (245965.64692868647 / (22))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (221) + (22) - 2, .975) and 0.052538111343040116 = (277526.15982355154 - 262945.45953813544) / 277526.15982355154 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.StaticDelegate #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 197.28386863839285 < 208.0596257346278. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/15/2023 8:45:09 AM, 10/10/2023 10:11:26 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 21.85334671491215 (T) = (0 -197023.2779219855) / Math.Sqrt((104559489.28530952 / (222)) + (101231.5659724759 / (20))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (222) + (20) - 2, .975) and 0.07108877554849599 = (212101.29960300802 - 197023.2779219855) / 212101.29960300802 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
325.60 ms 277.90 ms 0.85 0.08 False
323.53 ms 282.64 ms 0.87 0.10 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### LinqBenchmarks.Where00LinqMethodX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 277.9039305 < 307.6019702041667. IsChangePoint: Marked as a change because one of 9/28/2023 2:34:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 9.281375063133797 (T) = (0 -282521811.8812107) / Math.Sqrt((258263046499397.06 / (227)) + (24961202432997.96 / (15))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (15) - 2, .975) and 0.05212319414113991 = (298057521.96375453 - 282521811.8812107) / 298057521.96375453 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Where00LinqQueryX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 282.6375768333333 < 306.60040726339287. IsChangePoint: Marked as a change because one of 9/28/2023 2:34:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 12.291288359197171 (T) = (0 -281369722.2341626) / Math.Sqrt((273442436944964.6 / (227)) + (7702577818570.63 / (15))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (15) - 2, .975) and 0.054157990051294944 = (297480677.82421917 - 281369722.2341626) / 297480677.82421917 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
677.64 ns 571.65 ns 0.84 0.04 False
44.63 ns 33.27 ns 0.75 0.12 False
625.09 ns 544.40 ns 0.87 0.06 False
39.85 ns 35.84 ns 0.90 0.03 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 571.6482786146883 < 644.2747314009. IsChangePoint: Marked as a change because one of 8/19/2023 11:06:43 PM, 8/28/2023 1:23:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 20.824000239398465 (T) = (0 -571.7356308997989) / Math.Sqrt((3380.156786421331 / (229)) + (14.015213380797686 / (15))) is greater than 1.96981513413532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (15) - 2, .975) and 0.12609853931011886 = (654.2335224482352 - 571.7356308997989) / 654.2335224482352 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 33.270768804700744 < 42.370614236132894. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 40.813868499723014 (T) = (0 -33.205680861324275) / Math.Sqrt((11.065276695394088 / (230)) + (0.024584951598180426 / (15))) is greater than 1.9697743954265454 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (230) + (15) - 2, .975) and 0.21516259977603652 = (42.30899400544445 - 33.205680861324275) / 42.30899400544445 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 544.3970190796028 < 593.3579130041255. IsChangePoint: Marked as a change because one of 8/19/2023 11:06:43 PM, 8/28/2023 1:23:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 38.55139523385047 (T) = (0 -543.2203969778228) / Math.Sqrt((531.0333122585494 / (228)) + (17.231189399355607 / (15))) is greater than 1.9698562125959236 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (15) - 2, .975) and 0.11687959906733943 = (615.1147639711749 - 543.2203969778228) / 615.1147639711749 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.83809831951996 < 37.85133822388158. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 30.78752540339366 (T) = (0 -35.83394499725807) / Math.Sqrt((2.5761435089259224 / (227)) + (0.00855097411995181 / (15))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (15) - 2, .975) and 0.08575464187646453 = (39.19510739524705 - 35.83394499725807) / 39.19510739524705 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 7c82eaafe6c67cfd85cb213cd5647e50074fb48e
Compare 5280006fb69e15a3a429ff34a9a0b2f3c2434000
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Perf_Frozen<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.49 ns 9.55 ns 0.66 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.554277461040087 < 13.747484572280808. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsImprovementStdDev: Marked as improvement because 22.23929781961761 (T) = (0 -9.630314052831734) / Math.Sqrt((3.830581673118656 / (162)) + (0.012731224963993882 / (15))) is greater than 1.9736124619541988 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (162) + (15) - 2, .975) and 0.26547330073285186 = (13.11091082521587 - 9.630314052831734) / 13.11091082521587 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)