search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 16 Regressions on 10/10/2023 7:44:23 AM #23131

Open performanceautofiler[bot] opened 12 months ago

performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
23.95 ns 28.58 ns 1.19 0.15 False
4.24 ns 5.38 ns 1.27 0.04 False
12.30 ns 13.80 ns 1.12 0.18 False
19.10 ns 21.73 ns 1.14 0.05 False
17.21 ns 20.36 ns 1.18 0.12 False
12.89 ns 14.00 ns 1.09 0.25 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int64.ToString(value: -9223372036854775808) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.583902078847835 > 25.198803989811392. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/13/2023 1:12:35 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -11.24273045664896 (T) = (0 -28.399666059154125) / Math.Sqrt((1.5453239946469701 / (17)) + (0.12967408904033192 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.1404651510100294 = (24.90182714833728 - 28.399666059154125) / 24.90182714833728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.TryFormat(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.3802959180986445 > 4.471960002428734. IsChangePoint: Marked as a change because one of 9/28/2023 2:34:07 PM, 10/10/2023 10:11:26 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -406.7907190513541 (T) = (0 -5.374852974670438) / Math.Sqrt((4.790263844504171E-05 / (19)) + (0.00010043225153050598 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.2624030628961577 = (4.257636196112874 - 5.374852974670438) / 4.257636196112874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.Parse(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.804006766303626 > 12.899764976407203. IsChangePoint: Marked as a change because one of 9/13/2023 1:12:35 PM, 9/28/2023 2:34:07 PM, 10/10/2023 10:11:26 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -19.688868740641443 (T) = (0 -13.880028258942897) / Math.Sqrt((0.08386545694999799 / (19)) + (0.010988851927717522 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.11103173573081879 = (12.492917900146963 - 13.880028258942897) / 12.492917900146963 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.ToString(value: 9223372036854775807) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.73340118003498 > 20.037043657604645. IsChangePoint: Marked as a change because one of 8/27/2023 2:42:29 PM, 9/2/2023 3:43:46 AM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -14.583205080369117 (T) = (0 -21.35902590091399) / Math.Sqrt((0.11594312151248541 / (17)) + (0.29153923602435255 / (21))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (21) - 2, .025) and -0.10894247738591459 = (19.260715804901935 - 21.35902590091399) / 19.260715804901935 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.36027198005898 > 18.124344903678633. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/15/2023 8:45:09 AM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -20.62499947966154 (T) = (0 -19.64598396936502) / Math.Sqrt((0.07658350616076853 / (17)) + (0.19425246004074534 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.13795402467520676 = (17.26430377973517 - 19.64598396936502) / 17.26430377973517 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.ParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.004474197464821 > 13.447098694101555. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/28/2023 2:34:07 PM, 10/10/2023 10:11:26 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -21.716319479686522 (T) = (0 -14.030636850773757) / Math.Sqrt((0.011365481259168444 / (19)) + (0.05566478991060573 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.09890511846313176 = (12.767832832006674 - 14.030636850773757) / 12.767832832006674 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.18 ns 6.46 ns 1.55 0.16 False
12.74 ns 14.96 ns 1.17 0.29 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.459778594248588 > 4.342183347376697. IsChangePoint: Marked as a change because one of 8/27/2023 2:42:29 PM, 9/2/2023 3:43:46 AM, 9/14/2023 1:27:50 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -22.879047519150788 (T) = (0 -6.215941559134138) / Math.Sqrt((0.053132983806516224 / (17)) + (0.1087686995841994 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.49396264550311747 = (4.160707483446357 - 6.215941559134138) / 4.160707483446357 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.960902298381706 > 13.404396987791076. IsChangePoint: Marked as a change because one of 9/14/2023 1:27:50 PM, 10/10/2023 10:11:26 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -25.397359618186492 (T) = (0 -14.973449039766493) / Math.Sqrt((0.05405291122882074 / (19)) + (0.1108404671353817 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.18390054685198354 = (12.647556485704149 - 14.973449039766493) / 12.647556485704149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.89 μs 4.46 μs 1.54 0.16 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.4593207023809525 > 3.0323290046007427. IsChangePoint: Marked as a change because one of 8/11/2023 7:16:47 PM, 8/22/2023 4:48:44 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -9.3913601112413 (T) = (0 -4091.6304001314247) / Math.Sqrt((643.1369249491569 / (17)) + (350841.5792166327 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.40883881124436844 = (2904.2572986170494 - 4091.6304001314247) / 2904.2572986170494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.06 ns 21.05 ns 1.10 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.04872646415265 > 20.119323979222155. IsChangePoint: Marked as a change because one of 8/8/2023 9:37:01 AM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -15.453290986625055 (T) = (0 -20.93402998057023) / Math.Sqrt((0.14848781412419274 / (17)) + (0.02191731608398557 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.07853769771187766 = (19.409641429299935 - 20.93402998057023) / 19.409641429299935 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.44 ns 23.03 ns 1.13 0.20 False
14.66 ns 16.83 ns 1.15 0.29 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionSingleSegment #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.026349126597765 > 22.688347325305735. IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/19/2023 8:08:41 AM, 9/28/2023 2:34:07 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -6.034049034747316 (T) = (0 -23.152843278287833) / Math.Sqrt((1.0626913502157809 / (17)) + (0.04674670697712259 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.0709603043986341 = (21.61876885930765 - 23.152843278287833) / 21.61876885930765 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionArray #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.833741582672804 > 15.38104682595051. IsChangePoint: Marked as a change because one of 9/28/2023 2:34:07 PM, 10/10/2023 10:11:26 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -7.667409771107031 (T) = (0 -16.4375082334086) / Math.Sqrt((0.5586055546077049 / (19)) + (0.0629791011133453 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.09188856840944114 = (15.054199401824667 - 16.4375082334086) / 15.054199401824667 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.52 μs 13.22 μs 1.06 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.215886269425841 > 13.122914122648883. IsChangePoint: Marked as a change because one of 8/19/2023 11:06:43 PM, 8/22/2023 4:48:44 PM, 9/28/2023 2:34:07 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -12.808842432642393 (T) = (0 -13262.439221622899) / Math.Sqrt((32387.23960579843 / (17)) + (28238.40826514024 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.05768309877013888 = (12539.142619414362 - 13262.439221622899) / 12539.142619414362 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 94f212275b2f51ca67025d677d7d5c5bc75f670f
Compare 1d760d6de8250eecb5b3711957e0393a796c89b5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.00 ns 10.18 ns 1.13 0.21 False
12.18 ns 13.92 ns 1.14 0.17 False
4.44 ns 5.74 ns 1.29 0.27 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Char>.ToArray(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.175075903821932 > 9.46056919656343. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 10/10/2023 1:59:38 AM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -9.078217882506292 (T) = (0 -9.961247295707965) / Math.Sqrt((0.052298214559888465 / (17)) + (0.08721312104068431 / (22))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (22) - 2, .025) and -0.08280197660603236 = (9.199509708073126 - 9.961247295707965) / 9.199509708073126 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Char>.ToArray(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.921685171853053 > 12.872043947222478. IsChangePoint: Marked as a change because one of 9/18/2023 2:28:59 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -14.807673457477254 (T) = (0 -13.590830085602889) / Math.Sqrt((0.047365653595488895 / (16)) + (0.08252320854648555 / (23))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (23) - 2, .025) and -0.09669197307657787 = (12.392568213548776 - 13.590830085602889) / 12.392568213548776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.743219177159952 > 4.6572128622027265. IsChangePoint: Marked as a change because one of 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -15.804208044900031 (T) = (0 -5.734510798545107) / Math.Sqrt((0.13613941139287555 / (24)) + (0.00017024844136829387 / (15))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (15) - 2, .025) and -0.26227028252364737 = (4.54301339256767 - 5.734510798545107) / 4.54301339256767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)