dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 23 Regressions on 1/3/2023 9:28:36 AM #11469

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 160.84 ns 181.86 ns 1.13 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 181.86368195848553 > 163.09045853242492. IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -17.373447697885478 (T) = (0 -180.2959422172667) / Math.Sqrt((2.215138481741618 / (14)) + (62.7083020230811 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.15789350457435294 = (155.71029762667538 - 180.2959422172667) / 155.71029762667538 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateElementWithNamespace - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Xml.Linq.Perf_XElement.CreateElementWithNamespace.html>) 71.98 ns 78.65 ns 1.09 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XElement.CreateElementWithNamespace ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.65085528061793 > 75.54473792011882. IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -14.233663694493998 (T) = (0 -77.10413025393633) / Math.Sqrt((0.21062382707473465 / (14)) + (3.882905046283274 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.07108436058240564 = (71.98698169022906 - 77.10413025393633) / 71.98698169022906 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22The%22%2c%20Options%3a%20Compiled).html>) 62.28 μs 78.28 μs 1.26 0.07 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20NonBacktracking).html>) 3.88 ms 4.11 ms 1.06 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%22%2c%20Options%3a%20Compiled).html>) 417.59 μs 459.58 μs 1.10 0.08 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.27937291666666 > 65.40259312749004. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -48.284719766577496 (T) = (0 -77058.67421646418) / Math.Sqrt((34143.006617580206 / (14)) + (3055862.743202205 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.23515266429901255 = (62387.97554648628 - 77058.67421646418) / 62387.97554648628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.107238360655738 > 4.0752973925781255. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 1/5/2023 12:24:13 PM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -34.08492961305361 (T) = (0 -4106973.671147753) / Math.Sqrt((637545150.2304235 / (27)) + (295248946.1833007 / (21))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .025) and -0.053673167181667765 = (3897768.111655494 - 4106973.671147753) / 3897768.111655494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 459.5782352941177 > 427.38174679487184. IsChangePoint: Marked as a change because one of 10/28/2022 6:19:52 PM, 11/16/2022 6:51:59 PM, 11/18/2022 8:48:12 PM, 12/12/2022 5:25:39 PM, 12/20/2022 11:23:14 PM, 1/5/2023 11:35:29 PM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -17.220094804463244 (T) = (0 -457187.76544353116) / Math.Sqrt((11451784.410498466 / (30)) + (125757052.18693747 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.11388392699841657 = (410444.71004758566 - 457187.76544353116) / 410444.71004758566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in Benchstone.MDBenchI.MDNDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Benchstone.MDBenchI.MDNDhrystone.Test.html>) 342.07 ms 383.69 ms 1.12 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchI.MDNDhrystone.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 383.68524 > 357.44929874999997. IsChangePoint: Marked as a change because one of 10/18/2022 1:35:00 AM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 8:48:12 PM, 12/6/2022 2:12:43 AM, 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -11.346917916453949 (T) = (0 -377366201.5099116) / Math.Sqrt((4631784702645.482 / (14)) + (350919596129101.25 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.10881526318996217 = (340332798.4720041 - 377366201.5099116) / 340332798.4720041 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Get - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2010).html>) 6.61 ms 7.35 ms 1.11 0.24 False
[Get - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%208%2c%20KeysCountPerProvider%3a%2020).html>) 904.81 μs 953.11 μs 1.05 0.20 False
[Get - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%208%2c%20KeysCountPerProvider%3a%2040).html>) 2.00 ms 2.20 ms 1.10 0.21 False
[Get - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2040).html>) 7.40 ms 8.09 ms 1.09 0.15 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.3504567099567115 > 6.986854845114344. IsChangePoint: Marked as a change because one of 10/26/2022 10:56:11 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -10.883955028824111 (T) = (0 -7258501.954136837) / Math.Sqrt((23794173004.52431 / (14)) + (29390099538.238804 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.08216607409074052 = (6707382.654030795 - 7258501.954136837) / 6707382.654030795 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 20) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 953.1145911949685 > 948.3721489510489. IsChangePoint: Marked as a change because one of 10/24/2022 9:01:44 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -11.821767841596184 (T) = (0 -994046.5268137222) / Math.Sqrt((325032541.7090268 / (14)) + (987618849.4116402 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.09406334827113486 = (908582.2392136053 - 994046.5268137222) / 908582.2392136053 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 40) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1956925146198834 > 2.1105269576774197. IsChangePoint: Marked as a change because one of 10/25/2022 12:27:43 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -12.588639468983306 (T) = (0 -2170291.6012780373) / Math.Sqrt((990321816.9735658 / (14)) + (2511533295.016766 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.07498160707943334 = (2018910.4511047401 - 2170291.6012780373) / 2018910.4511047401 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 40) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.088875909090909 > 7.7480420497159095. IsChangePoint: Marked as a change because one of 10/26/2022 10:56:11 AM, 1/2/2023 8:42:52 PM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -14.990579572580764 (T) = (0 -8074643.286204915) / Math.Sqrt((15120757115.712225 / (12)) + (30059657248.247192 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.09286665901901295 = (7388498.1480292715 - 8074643.286204915) / 7388498.1480292715 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_Encoding.GetBytes(size%3a%20512%2c%20encName%3a%20%22utf-8%22).html>) 77.59 ns 82.65 ns 1.07 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.65231190953013 > 81.5011748090793. IsChangePoint: Marked as a change because one of 11/23/2022 8:02:34 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -11.147228470052537 (T) = (0 -83.53735935808486) / Math.Sqrt((0.4265207088942191 / (14)) + (8.886719172036917 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.07768753119262238 = (77.51538079468942 - 83.53735935808486) / 77.51538079468942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyThreeValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Byte).IndexOfAnyThreeValues(Size%3a%20512).html>) 8.84 ns 10.61 ns 1.20 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.605357442452673 > 9.284810117728103. IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 12/12/2022 5:25:39 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -11.757572159589321 (T) = (0 -9.858734050546374) / Math.Sqrt((1.1705080126533855E-05 / (14)) + (0.25325774571375786 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.11474633902606857 = (8.843925927722491 - 9.858734050546374) / 8.843925927722491 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).ConcurrentDictionary(Size%3a%20512).html>) 3.75 μs 4.07 μs 1.09 0.02 False
[IDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).IDictionary(Size%3a%20512).html>) 3.85 μs 4.25 μs 1.10 0.03 False
[SortedList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 19.08 μs 22.01 μs 1.15 0.08 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.071979030856149 > 3.946716957486198. IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 12/12/2022 5:25:39 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -17.14744104847785 (T) = (0 -4056.4116834054253) / Math.Sqrt((891.3337414895375 / (14)) + (7436.461978742675 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.07646807477832537 = (3768.260089125962 - 4056.4116834054253) / 3768.260089125962 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<Int32, Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.248849936834506 > 4.050559376965712. IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 12/12/2022 5:25:39 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -17.676392841332014 (T) = (0 -4248.738658927083) / Math.Sqrt((451.5444881063704 / (14)) + (14343.773947146794 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.0972842004904755 = (3872.0494262360994 - 4248.738658927083) / 3872.0494262360994 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.007302508204408 > 19.704773378380512. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/26/2022 7:11:30 AM, 11/18/2022 8:48:12 PM, 12/3/2022 7:24:45 AM, 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -15.323142867745583 (T) = (0 -21238.334679818858) / Math.Sqrt((101764.4253976825 / (14)) + (715510.6902482156 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.1381758528011608 = (18659.97651202076 - 21238.334679818858) / 18659.97651202076 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Char).SequenceEqual(Size%3a%20512).html>) 15.27 ns 20.06 ns 1.31 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.SequenceEqual(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.056871753662683 > 16.074562964566585. IsChangePoint: Marked as a change because one of 10/31/2022 10:23:25 PM, 12/14/2022 5:20:21 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -18.06134337115146 (T) = (0 -19.61206076739156) / Math.Sqrt((0.0052690868689119884 / (14)) + (1.9261003540234498 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.2819197783639026 = (15.298976658603534 - 19.61206076739156) / 15.298976658603534 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAny - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny.html>) 6.29 ns 7.58 ns 1.20 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.578208364140196 > 6.648163091472983. IsChangePoint: Marked as a change because one of 11/4/2022 4:27:34 PM, 11/18/2022 7:31:41 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -92.66458962284825 (T) = (0 -7.561091216951475) / Math.Sqrt((0.0006941602791081872 / (14)) + (0.004370574226628063 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.195553652313157 = (6.324342870202669 - 7.561091216951475) / 6.324342870202669 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Replace_Char - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Replace_Char(text%3a%20%22This%20is%20a%20very%20nice%20sentence%22%2c%20oldChar%3a%20%27i%27%2c%20newChar%3a%20%27I%27).html>) 17.60 ns 20.71 ns 1.18 0.06 False
[IndexerCheckBoundCheckHoist - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.IndexerCheckBoundCheckHoist.html>) 27.29 ns 30.10 ns 1.10 0.07 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.712834154487364 > 18.564739532119972. IsChangePoint: Marked as a change because one of 12/13/2022 4:26:35 PM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -19.982855418673594 (T) = (0 -20.678749897592773) / Math.Sqrt((0.024924312859676057 / (14)) + (0.7551074038184491 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.17601770785943083 = (17.583706231117823 - 20.678749897592773) / 17.583706231117823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.IndexerCheckBoundCheckHoist ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.100800665638474 > 28.46043890989043. IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM, 11/16/2022 6:51:59 PM, 12/1/2022 7:29:59 PM, 12/14/2022 5:20:21 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -8.25801545845388 (T) = (0 -29.730920223649044) / Math.Sqrt((0.9291895484318832 / (14)) + (0.2087665854249347 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.0808408085734506 = (27.50721474227962 - 29.730920223649044) / 27.50721474227962 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Add - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_HashCode.Add.html>) 2.12 μs 2.27 μs 1.07 0.18 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_HashCode.Add ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.265429134929271 > 2.228025747787463. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -14.315533556842976 (T) = (0 -2339.4445502758967) / Math.Sqrt((17.62558060701369 / (14)) + (7663.2329192076695 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.10147152040850582 = (2123.9264991692753 - 2339.4445502758967) / 2123.9264991692753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CreateAddAndClear(Int32).ICollection(Size%3a%20512).html>) 1.49 μs 1.66 μs 1.11 0.31 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6609100871620248 > 1.5701686432163309. IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 4:41:14 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -8.184321392116432 (T) = (0 -1626.4451362654538) / Math.Sqrt((626.6891453092213 / (14)) + (6130.812902603171 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.0816588991860211 = (1503.658073251558 - 1626.4451362654538) / 1503.658073251558 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.AddGivenSize(Int32).ICollection(Size%3a%20512).html>) 1.36 μs 1.64 μs 1.21 0.17 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.640530520974997 > 1.4262391190866712. IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 8:48:12 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -12.17145466383382 (T) = (0 -1567.5940304032888) / Math.Sqrt((12.839101098106028 / (14)) + (9898.777186355532 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.1529922847199838 = (1359.5876149197263 - 1567.5940304032888) / 1359.5876149197263 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in Benchstone.BenchI.EightQueens

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Benchstone.BenchI.EightQueens.Test.html>) 1.78 μs 1.90 μs 1.06 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.EightQueens.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8966301806160297 > 1.85449661668785. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 11/11/2022 5:53:56 PM, 12/6/2022 2:12:43 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM. IsRegressionStdDev: Marked as regression because -13.871060847089696 (T) = (0 -1892.788948494281) / Math.Sqrt((290.13806680477796 / (14)) + (2359.29819183111 / (34))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (34) - 2, .025) and -0.07476904455832853 = (1761.1122669355573 - 1892.788948494281) / 1761.1122669355573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)