search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 6/7/2022 2:18:03 AM #6016

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.AddGivenSize(Int32).ICollection(Size%3a%20512).html>) 981.67 ns 1.09 μs 1.11 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0942717150862238 > 1031.848803721274. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -223.22852773782023 (T) = (0 -1095.308780402469) / Math.Sqrt((3.3765038488305175 / (27)) + (6.005282322884887 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1141772875846443 = (983.0650764537849 - 1095.308780402469) / 983.0650764537849 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Concat_CharEnumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Concat_CharEnumerable.html>) 5.19 μs 5.85 μs 1.13 0.03 False
[TrimStart - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.TrimStart(s%3a%20%22%20Test%22).html>) 10.42 ns 11.57 ns 1.11 0.10 False
[Format_OneArg - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Format_OneArg(s%3a%20%22Testing%20%7b0%7d%2c%20%7b0%3aC%7d%2c%20%7b0%3aD5%7d%2c%20%7b0%3aE%7d%20-%20%7b0%3aF4%7d%7b0%3aG%7d%7b0%3aN%7d%20%20%7b0%3aX%7d%20!!%22%2c%20o%3a%208).html>) 379.03 ns 414.40 ns 1.09 0.02 False
[Insert - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Insert(s1%3a%20%22Test%22%2c%20i%3a%202%2c%20s2%3a%20%22%20Test%22).html>) 11.59 ns 12.73 ns 1.10 0.07 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Concat_CharEnumerable ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.854842211110813 > 5.445431158577336. IsChangePoint: Marked as a change because one of 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -358.90965411858747 (T) = (0 -5847.735339054095) / Math.Sqrt((22.41030980758999 / (27)) + (119.1472158524296 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.12687283958702514 = (5189.348020134344 - 5847.735339054095) / 5189.348020134344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.TrimStart(s: " Test") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.57246994208824 > 10.950476650578084. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -7.527032312238632 (T) = (0 -11.461006722745145) / Math.Sqrt((0.11750420375239466 / (27)) + (0.5137026847539943 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08836130008961184 = (10.530516586542985 - 11.461006722745145) / 10.530516586542985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N} {0:X} !!", o: 8) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 414.39570720996954 > 402.31498951458525. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/18/2022 10:04:15 AM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -31.981591874787 (T) = (0 -413.29701690406415) / Math.Sqrt((10.37366182016404 / (24)) + (22.568991335148297 / (50))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (50) - 2, .025) and -0.0784454403985838 = (383.23405285232906 - 413.29701690406415) / 383.23405285232906 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.72766878587897 > 12.16915336876742. IsChangePoint: Marked as a change because one of 5/13/2022 2:40:35 AM, 5/13/2022 11:25:51 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -5.8748685927677995 (T) = (0 -12.604475728429044) / Math.Sqrt((0.40795672243369474 / (27)) + (0.04385097211666746 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.06273877344549354 = (11.860370622937012 - 12.604475728429044) / 11.860370622937012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.Json.Tests.Perf_Get.GetInt64.html>) 407.07 ns 469.35 ns 1.15 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Get.GetInt64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 469.35341255010695 > 426.9852611963788. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/12/2022 3:44:01 PM, 5/20/2022 10:18:01 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -650.1712579831699 (T) = (0 -469.31527109219843) / Math.Sqrt((0.12347566229666719 / (27)) + (0.21623765699450723 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.15395152223588957 = (406.7027618134737 - 469.31527109219843) / 406.7027618134737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEach(Int32).ImmutableQueue(Size%3a%20512).html>) 2.83 μs 3.11 μs 1.10 0.10 False
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEach(Int32).ConcurrentStack(Size%3a%20512).html>) 1.81 μs 2.03 μs 1.12 0.36 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.ImmutableQueue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1105380499867676 > 2.975029829825203. IsChangePoint: Marked as a change because one of 4/16/2022 10:12:41 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -24.752942115846338 (T) = (0 -3264.4093799396373) / Math.Sqrt((2819.6783047625026 / (27)) + (6968.284114817723 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1370572145135812 = (2870.9279869756692 - 3264.4093799396373) / 2870.9279869756692 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.029215820907068 > 1.9051555061409193. IsChangePoint: Marked as a change because one of 4/14/2022 4:58:13 PM, 5/21/2022 4:54:11 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -475.6739753411041 (T) = (0 -2033.019298809442) / Math.Sqrt((3.738903352674159 / (27)) + (3.4647240889628503 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.12079909502340855 = (1813.901624150563 - 2033.019298809442) / 1813.901624150563 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IndexerSet(String).Span(Size%3a%20512).html>) 180.40 ns 231.66 ns 1.28 0.18 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 231.65834979173619 > 189.4119391785876. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -13.595570552892323 (T) = (0 -238.90473356466788) / Math.Sqrt((261.50618809620505 / (27)) + (276.30718685088914 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.289508706215627 = (185.26802681758636 - 238.90473356466788) / 185.26802681758636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(Int32).Span(Size%3a%20512).html>) 15.00 μs 17.07 μs 1.14 0.05 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(Int32).Queue(Size%3a%20512).html>) 17.58 μs 19.01 μs 1.08 0.32 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.074695180180182 > 15.766185410212538. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -69.79325752761652 (T) = (0 -17039.68909028491) / Math.Sqrt((10760.357606283698 / (27)) + (19064.508887837885 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.13222853658141315 = (15049.690534857553 - 17039.68909028491) / 15049.690534857553 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.008960525781568 > 18.451447655873032. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -11.944918529430971 (T) = (0 -18953.86297128183) / Math.Sqrt((329080.43650576536 / (27)) + (192804.82689748536 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08747206257535868 = (17429.287264994342 - 18953.86297128183) / 17429.287264994342 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Numerics.Tests.Perf_VectorOf(Int16).GetHashCodeBenchmark.html>) 34.20 ns 38.48 ns 1.13 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.47779519665282 > 35.865069697205385. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -170.85402341246942 (T) = (0 -38.39640507765742) / Math.Sqrt((0.011545529063144195 / (27)) + (0.010262295155936144 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1275143840487099 = (34.054026822950625 - 38.39640507765742) / 34.054026822950625 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/MicroBenchmarks.Serializers.JsonToStream(IndexViewModel).DataContractJsonSerializer.html>) 61.67 μs 66.41 μs 1.08 0.18 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.40937988307202 > 64.02082140913036. IsChangePoint: Marked as a change because one of 4/22/2022 6:17:37 AM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -12.856769239693497 (T) = (0 -66042.13734868835) / Math.Sqrt((2710414.47820309 / (27)) + (764803.2579469321 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.07122789496874743 = (61650.87527954554 - 66042.13734868835) / 61650.87527954554 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_UInt32.TryParse(value%3a%20%224294967295%22).html>) 14.90 ns 17.20 ns 1.15 0.32 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_UInt32.Parse(value%3a%20%224294967295%22).html>) 14.48 ns 16.18 ns 1.12 0.33 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt32.TryParse(value: "4294967295") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.202942577248162 > 15.653638458424739. IsChangePoint: Marked as a change because one of 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -14.913623000799644 (T) = (0 -17.11369668138154) / Math.Sqrt((0.36793753703809234 / (27)) + (0.2688572389373074 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1379338389531662 = (15.039272140043888 - 17.11369668138154) / 15.039272140043888 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.Parse(value: "4294967295") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.18298789480534 > 15.627847588185597. IsChangePoint: Marked as a change because one of 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -9.55505997091883 (T) = (0 -17.108557799769567) / Math.Sqrt((0.8801859725481722 / (27)) + (0.47883696698193906 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.13061505624531378 = (15.132080282555037 - 17.108557799769567) / 15.132080282555037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateFor(String).ReadOnlySpan(Size%3a%20512).html>) 174.18 ns 228.32 ns 1.31 0.04 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateFor(String).Span(Size%3a%20512).html>) 179.81 ns 231.57 ns 1.29 0.04 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 228.31776919855324 > 182.91253393826057. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -624.0499049605817 (T) = (0 -228.37888012889152) / Math.Sqrt((0.19394928121114263 / (27)) + (0.01458959920436933 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.3098334454081987 = (174.3571909310341 - 228.37888012889152) / 174.3571909310341 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 231.57313348927283 > 188.79996496791867. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -468.0610989864261 (T) = (0 -231.51829173832792) / Math.Sqrt((0.24101635153834325 / (27)) + (0.1509607938860292 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.2865715200943692 = (179.94980311809343 - 231.51829173832792) / 179.94980311809343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WhereSelect - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Linq.Tests.Perf_Enumerable.WhereSelect(input%3a%20IEnumerable).html>) 986.93 ns 1.07 μs 1.08 0.03 False
[SelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Range).html>) 192.76 ns 214.51 ns 1.11 0.04 False
[WhereFirst_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input%3a%20Array).html>) 187.84 ns 210.70 ns 1.12 0.03 False
[EmptyTakeSelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray.html>) 13.41 ns 15.06 ns 1.12 0.04 False
[WhereAny_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input%3a%20Array).html>) 186.05 ns 208.74 ns 1.12 0.03 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0702990378020472 > 1036.8144769596536. IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -132.33931219950136 (T) = (0 -1067.4858177910048) / Math.Sqrt((3.472420815078482 / (27)) + (11.410786971098128 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08172587821446027 = (986.8357957313912 - 1067.4858177910048) / 986.8357957313912 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.506810466609 > 201.88424871116896. IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -105.39625351917039 (T) = (0 -214.3633381639935) / Math.Sqrt((0.5657009080986409 / (27)) + (0.9993888767642436 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.11237270765904962 = (192.7081963518449 - 214.3633381639935) / 192.7081963518449 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 210.70243795076115 > 196.97639036301973. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -112.46027019585667 (T) = (0 -210.41419809080222) / Math.Sqrt((0.6191395906443802 / (27)) + (0.7418010793701387 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.11810164778364458 = (188.18879169697627 - 210.41419809080222) / 188.18879169697627 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.056934826963339 > 14.193929966618821. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -33.81822113284151 (T) = (0 -15.506314145099564) / Math.Sqrt((0.0462500738344724 / (27)) + (0.09043657418383935 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.15144939052683382 = (13.466778716175106 - 15.506314145099564) / 13.466778716175106 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 208.7391529891834 > 194.97314246056996. IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -163.93626073181227 (T) = (0 -208.62296124141838) / Math.Sqrt((0.22026318991981111 / (27)) + (0.5412532349670869 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1238726718586913 = (185.6286450104637 - 208.62296124141838) / 185.6286450104637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsFalse(String).ImmutableArray(Size%3a%20512).html>) 656.90 μs 769.95 μs 1.17 0.04 False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsFalse(String).LinkedList(Size%3a%20512).html>) 966.35 μs 1.06 ms 1.10 0.06 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsFalse(String).Array(Size%3a%20512).html>) 488.05 μs 541.69 μs 1.11 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 769.9460761408732 > 690.5275573242188. IsChangePoint: Marked as a change because one of 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -427.9814382496859 (T) = (0 -764903.5499538758) / Math.Sqrt((858025.9529665317 / (27)) + (1438712.274284132 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.16246263554690588 = (658002.6975181102 - 764903.5499538758) / 658002.6975181102 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0640288541666667 > 1000.8732602389706. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 5/11/2022 11:37:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -30.093377364070946 (T) = (0 -1063776.6738262477) / Math.Sqrt((320615848.93987 / (27)) + (6095906.566419081 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.10866517989258401 = (959511.2150354668 - 1063776.6738262477) / 959511.2150354668 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 541.6931694889163 > 512.6308444229404. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -15.09327265557407 (T) = (0 -538381.6637188481) / Math.Sqrt((247779748.27827236 / (27)) + (5723580.714272892 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.09347973919340374 = (492356.3230499185 - 538381.6637188481) / 492356.3230499185 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Numerics.Tests.Perf_VectorOf(SByte).GetHashCodeBenchmark.html>) 71.13 ns 80.29 ns 1.13 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<SByte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.29352985459032 > 74.63929471092885. IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -423.7880265707549 (T) = (0 -80.29063059107267) / Math.Sqrt((0.003432043380623234 / (27)) + (0.016368166707542403 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.13004584638750036 = (71.05077271665 - 80.29063059107267) / 71.05077271665 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsFalse(Int32).Array(Size%3a%20512).html>) 19.30 μs 21.92 μs 1.14 0.13 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.918159356007482 > 19.869875472543615. IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -28.489464098690043 (T) = (0 -21711.32856367154) / Math.Sqrt((20605.162054506785 / (27)) + (394196.8518077276 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1435379094333236 = (18986.10302690404 - 21711.32856367154) / 18986.10302690404 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%202147483647).html>) 12.90 ns 14.17 ns 1.10 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.172989261414129 > 13.557483332318958. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -41.966330674047775 (T) = (0 -14.011021743417807) / Math.Sqrt((0.00334267528513837 / (27)) + (0.025118578291016724 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08324335299929524 = (12.934325149214114 - 14.011021743417807) / 12.934325149214114 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/Benchstone.BenchI.Fib.Test.html>) 104.55 μs 111.27 μs 1.06 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Fib*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.Fib.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 111.27178742401216 > 109.77462644615385. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -54.714311205884904 (T) = (0 -111302.57817642957) / Math.Sqrt((389421.94965301227 / (27)) + (7676.877557253959 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.06311763099619182 = (104694.50880250547 - 111302.57817642957) / 104694.50880250547 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ctor_seeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_Random.ctor_seeded.html>) 266.79 ns 287.91 ns 1.08 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.ctor_seeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 287.91324109666033 > 279.5536557300006. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 11:38:14 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -124.7136055471177 (T) = (0 -288.2066429033205) / Math.Sqrt((0.6080356334038375 / (27)) + (0.38240389068082004 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08197582496101354 = (266.3706861599291 - 288.2066429033205) / 266.3706861599291 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEachNonGeneric(Int32).Stack(Size%3a%20512).html>) 2.75 μs 2.93 μs 1.07 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEachNonGeneric<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9311541346772247 > 2.8850412295553696. IsChangePoint: Marked as a change because one of 4/15/2022 3:46:08 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -243.41371744658983 (T) = (0 -2931.3410427106287) / Math.Sqrt((7.0956682214401905 / (27)) + (14.073609876328703 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.06639870003933977 = (2748.822783263418 - 2931.3410427106287) / 2748.822783263418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in Benchstone.BenchI.Ackermann

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/Benchstone.BenchI.Ackermann.Test.html>) 2.48 μs 2.67 μs 1.07 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Ackermann*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.Ackermann.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6652038893629126 > 2.634020971475332. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -29.007381909363374 (T) = (0 -2681.7638843889704) / Math.Sqrt((729.0615708665671 / (27)) + (761.4833258957955 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.07653847362657039 = (2491.0989714606526 - 2681.7638843889704) / 2491.0989714606526 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEachNonGeneric(String).Stack(Size%3a%20512).html>) 2.75 μs 2.94 μs 1.07 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.937592639144833 > 2.8912614924684203. IsChangePoint: Marked as a change because one of 4/14/2022 8:50:18 PM, 5/9/2022 11:56:12 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -185.93036135119192 (T) = (0 -2926.44583899465) / Math.Sqrt((19.025303391175456 / (27)) + (6.824417858480027 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.06221439889421282 = (2755.0425244104586 - 2926.44583899465) / 2755.0425244104586 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in Benchstone.BenchI.TreeInsert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/Benchstone.BenchI.TreeInsert.Test.html>) 46.27 μs 49.32 μs 1.07 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.TreeInsert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.TreeInsert.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.32074479166666 > 48.560431332285496. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -62.0720586065262 (T) = (0 -49419.60851254747) / Math.Sqrt((2210.5024898651554 / (27)) + (118605.63489248387 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.06927009913832903 = (46218.077689044374 - 49419.60851254747) / 46218.077689044374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CoveredIndex2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/Span.IndexerBench.CoveredIndex2(length%3a%201024).html>) 625.13 ns 679.59 ns 1.09 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Span.IndexerBench.CoveredIndex2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 679.5895142228063 > 656.508146351725. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -637.8309725028215 (T) = (0 -679.4747014323782) / Math.Sqrt((0.16706536157919116 / (27)) + (0.04995128029195178 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08687457683123133 = (625.163855992821 - 679.4747014323782) / 625.163855992821 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(String).Stack(Size%3a%20512).html>) 336.75 μs 381.49 μs 1.13 0.05 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(String).Array(Size%3a%20512).html>) 251.63 μs 273.12 μs 1.09 0.03 False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(String).LinkedList(Size%3a%20512).html>) 479.08 μs 532.07 μs 1.11 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 381.4894563789868 > 350.65684387737775. IsChangePoint: Marked as a change because one of 4/26/2022 12:44:27 PM, 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -62.77911053128988 (T) = (0 -381929.44764309295) / Math.Sqrt((14390163.518378617 / (27)) + (493598.6679217578 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.13788557701354517 = (335648.37744537694 - 381929.44764309295) / 335648.37744537694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 273.1154198737684 > 262.4863361576916. IsChangePoint: Marked as a change because one of 4/14/2022 8:50:18 PM, 5/9/2022 3:10:19 PM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -45.18876359962095 (T) = (0 -271320.1525124963) / Math.Sqrt((4844492.7986787995 / (27)) + (2842718.665567056 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.08882379213984042 = (249186.46568080314 - 271320.1525124963) / 249186.46568080314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 532.0705261111111 > 504.1941234564394. IsChangePoint: Marked as a change because one of 4/14/2022 8:50:18 PM, 5/9/2022 11:56:12 AM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -29.543653238397898 (T) = (0 -532685.9978253738) / Math.Sqrt((78333825.21714287 / (27)) + (1094779.0109959391 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.10478521128131647 = (482162.49854355946 - 532685.9978253738) / 482162.49854355946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.AddGivenSize(String).Stack(Size%3a%20512).html>) 1.50 μs 1.66 μs 1.11 0.27 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6600613182665729 > 1.5726792829687268. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -27.107344746584612 (T) = (0 -1701.6241949651053) / Math.Sqrt((157.95647819147968 / (27)) + (2269.473145679327 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.13277409086132555 = (1502.1743600008049 - 1701.6241949651053) / 1502.1743600008049 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Numerics.Tests.Perf_VectorOf(UInt16).GetHashCodeBenchmark.html>) 33.90 ns 38.34 ns 1.13 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.33761186578809 > 35.640846816907974. IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -179.06788182617495 (T) = (0 -38.47337898809894) / Math.Sqrt((0.003344197617616771 / (27)) + (0.024128980591099934 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1331346122119711 = (33.95305250891222 - 38.47337898809894) / 33.95305250891222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Numerics.Tests.Perf_VectorOf(Byte).GetHashCodeBenchmark.html>) 70.97 ns 80.72 ns 1.14 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.71760234171802 > 74.59563086919137. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -121.0185259970362 (T) = (0 -80.29570829731361) / Math.Sqrt((0.14073559437838112 / (27)) + (0.021980755654679555 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.1281453221636048 = (71.17496896881964 - 80.29570829731361) / 71.17496896881964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEach(String).ImmutableQueue(Size%3a%20512).html>) 3.24 μs 3.51 μs 1.08 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.ImmutableQueue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5086193507557137 > 3.44155594110601. IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -7.839795272496994 (T) = (0 -3735.6771708424185) / Math.Sqrt((9840.133150370168 / (27)) + (123114.93319656928 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.12948152788642095 = (3307.426530323985 - 3735.6771708424185) / 3307.426530323985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)