search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/16/2022 10:12:41 PM #5068

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 116.78 ns 226.90 ns 1.94 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 226.9007376447516 > 122.6767321460387. IsChangePoint: Marked as a change because one of 2/23/2022 6:59:06 PM, 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -2760.9086677895084 (T) = (0 -226.92607970236824) / Math.Sqrt((0.06603185550383836 / (46)) + (0.0017012101934099125 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.9423469705041189 = (116.83086654876682 - 226.92607970236824) / 116.83086654876682 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.Concurrent.AddRemoveFromDifferentThreads(String).ConcurrentQueue(Size%3a%202000000).html>) 24.06 ms 32.05 ms 1.33 0.52 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>.ConcurrentQueue(Size: 2000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.0515883 > 24.280855423125. IsChangePoint: Marked as a change because one of 2/24/2022 12:54:34 PM, 3/14/2022 9:01:19 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -6.6120056213276825 (T) = (0 -31806596.828468908) / Math.Sqrt((18943918669435.004 / (46)) + (9871111561570.65 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.31209232448600166 = (24241127.118039355 - 31806596.828468908) / 24241127.118039355 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Trim(s%3a%20%22%20Test%22).html>) 10.55 ns 11.94 ns 1.13 0.14 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Trim(s%3a%20%22%20Te%20st%20%20%22).html>) 13.27 ns 14.80 ns 1.11 0.06 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Trim(s%3a%20%22Test%20%22).html>) 10.76 ns 12.05 ns 1.12 0.10 False
[TrimStart - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.TrimStart(s%3a%20%22%20Test%22).html>) 10.28 ns 11.95 ns 1.16 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Trim(s: " Test") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.938674490618524 > 11.200035366850665. IsChangePoint: Marked as a change because one of 2/25/2022 5:57:13 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -10.472758061236597 (T) = (0 -11.91862347115635) / Math.Sqrt((0.04283604934959887 / (46)) + (0.23237029675301318 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.15008060358461325 = (10.363294045659016 - 11.91862347115635) / 10.363294045659016 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Trim(s: " Te st ") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.797898187448453 > 13.984379895623395. IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -35.03087003076377 (T) = (0 -14.786953896362554) / Math.Sqrt((0.08810576309222704 / (46)) + (0.0034671695045072346 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.12598204985964162 = (13.132495227793205 - 14.786953896362554) / 13.132495227793205 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Trim(s: "Test ") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.053331851479438 > 11.433688825056453. IsChangePoint: Marked as a change because one of 2/24/2022 9:06:13 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -34.05403580405217 (T) = (0 -12.139535921343105) / Math.Sqrt((0.030276203231365307 / (46)) + (0.012064260147921113 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.13316478079228689 = (10.712948484734389 - 12.139535921343105) / 10.712948484734389 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.TrimStart(s: " Test") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.950812547755511 > 10.79309906278997. IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 4/2/2022 3:40:16 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -14.889943993101932 (T) = (0 -11.59547902114433) / Math.Sqrt((0.05392207918309299 / (46)) + (0.1117709377822395 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.15835111471837598 = (10.010331818917859 - 11.59547902114433) / 10.010331818917859 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_Boolean.TryParse(value%3a%20%22false%22).html>) 2.95 ns 4.15 ns 1.41 0.01 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_Boolean.TryParse(value%3a%20%22False%22).html>) 2.95 ns 4.16 ns 1.41 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Boolean.TryParse(value: "false") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.148303603474224 > 3.098015135407483. IsChangePoint: Marked as a change because one of 2/8/2022 11:58:21 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -1136.0020196335684 (T) = (0 -4.153779206131033) / Math.Sqrt((5.770776123963784E-06 / (46)) + (1.0966556572170448E-05 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.40793943806305827 = (2.950254175595439 - 4.153779206131033) / 2.950254175595439 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.TryParse(value: "False") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.155272208792124 > 3.099267660750911. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/8/2022 11:58:21 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -620.2968343528779 (T) = (0 -4.155860483121808) / Math.Sqrt((0.00014166940770667279 / (46)) + (7.555505035106985E-06 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.4078119582643355 = (2.951999703316549 - 4.155860483121808) / 2.951999703316549 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsFalse(String).Array(Size%3a%20512).html>) 485.89 μs 535.57 μs 1.10 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 535.5687054166667 > 510.8680233637456. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/19/2022 11:51:48 PM, 2/23/2022 6:59:06 PM, 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/14/2022 4:00:26 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -36.68432197211849 (T) = (0 -538645.4987413066) / Math.Sqrt((6740271.210843837 / (46)) + (19317703.938823342 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.10368232184669923 = (488043.9670720069 - 538645.4987413066) / 488043.9670720069 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.TryGetValueFalse(String%2c%20String).IDictionary(Size%3a%20512).html>) 8.33 μs 10.15 μs 1.22 0.01 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.TryGetValueFalse(String%2c%20String).Dictionary(Size%3a%20512).html>) 7.61 μs 9.04 μs 1.19 0.00 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<String, String>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.152635144106219 > 8.782365383657185. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -57.398790659994624 (T) = (0 -10085.722866941116) / Math.Sqrt((3745.060874051329 / (46)) + (9210.274335290123 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.2084570769211336 = (8345.950435109522 - 10085.722866941116) / 8345.950435109522 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.TryGetValueFalse<String, String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.043817840713052 > 8.010285376607378. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -40.4058070579155 (T) = (0 -9008.503563649987) / Math.Sqrt((5794.528993533562 / (46)) + (10930.488841572553 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.1765857382257541 = (7656.478632177254 - 9008.503563649987) / 7656.478632177254 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BubbleSortArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/Span.Sorting.BubbleSortArray(Size%3a%20512).html>) 144.19 μs 151.71 μs 1.05 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.Sorting*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Span.Sorting.BubbleSortArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.70555542857144 > 151.386540725. IsChangePoint: Marked as a change because one of 3/6/2022 1:47:20 PM, 3/9/2022 2:35:55 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -32.754521014946164 (T) = (0 -151751.25782001333) / Math.Sqrt((3227947.4514640383 / (46)) + (4282.860215508543 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.060822978992230395 = (143050.5002485667 - 151751.25782001333) / 143050.5002485667 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.TryGetValueTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.TryGetValueTrue(String%2c%20String).IDictionary(Size%3a%20512).html>) 10.38 μs 12.24 μs 1.18 0.01 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.TryGetValueTrue(String%2c%20String).Dictionary(Size%3a%20512).html>) 9.78 μs 12.01 μs 1.23 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<String, String>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.239603924662124 > 11.164492989068425. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -35.5965631259579 (T) = (0 -12387.271198156424) / Math.Sqrt((15828.184885936735 / (46)) + (25104.080690521205 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.17269885012258407 = (10563.045403226552 - 12387.271198156424) / 10563.045403226552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.TryGetValueTrue<String, String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.012891595842882 > 10.333787910923453. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -42.75350088159367 (T) = (0 -11905.650683167747) / Math.Sqrt((29455.27181408835 / (46)) + (15361.080076496253 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.19341150351488232 = (9976.148753470834 - 11905.650683167747) / 9976.148753470834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForeachOverList100Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.LowLevelPerf.ForeachOverList100Elements.html>) 6.15 ms 10.51 ms 1.71 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ForeachOverList100Elements ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.508131235897439 > 6.460709094218751. IsChangePoint: Marked as a change because one of 3/15/2022 11:34:41 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -14.468396769996337 (T) = (0 -10275307.406177158) / Math.Sqrt((334158393904.3621 / (46)) + (619183493689.1334 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.55032418984029 = (6627844.339599507 - 10275307.406177158) / 6627844.339599507 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(Int32).Span(Size%3a%20512).html>) 14.17 μs 17.16 μs 1.21 0.18 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(Int32).Array(Size%3a%20512).html>) 15.60 μs 18.12 μs 1.16 0.20 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsTrue(Int32).ImmutableArray(Size%3a%20512).html>) 16.24 μs 18.22 μs 1.12 0.40 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.162787104166668 > 14.860787836402583. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -56.37745256859708 (T) = (0 -17309.477641302343) / Math.Sqrt((137420.62176312337 / (45)) + (3240.0922790222303 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.23223663543621278 = (14047.202577429272 - 17309.477641302343) / 14047.202577429272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.120819354204603 > 16.47811771003948. IsChangePoint: Marked as a change because one of 2/12/2022 1:03:18 PM, 2/19/2022 11:51:48 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -3.754539523158522 (T) = (0 -19692.48542669895) / Math.Sqrt((67524.00555975518 / (46)) + (12301418.98113097 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.25274672042341356 = (15719.446800900922 - 19692.48542669895) / 15719.446800900922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.216926046511627 > 17.045126678226854. IsChangePoint: Marked as a change because one of 4/1/2022 5:08:23 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -16.277176572696742 (T) = (0 -18933.13938566031) / Math.Sqrt((387230.7914233105 / (46)) + (323884.6124397518 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.2008921081342669 = (15765.895418427941 - 18933.13938566031) / 15765.895418427941 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LeadingZeroCount_uint - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint.html>) 371.87 ns 439.94 ns 1.18 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 439.9361941986373 > 391.12170897867327. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 3/15/2022 11:34:41 AM, 3/16/2022 12:48:00 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -333.2420629413616 (T) = (0 -439.8230492527828) / Math.Sqrt((0.1248576981618643 / (46)) + (0.42290696743085754 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.18163764656359158 = (372.2148245122902 - 439.8230492527828) / 372.2148245122902 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IndexerSetReverse(String).IList(Size%3a%20512).html>) 1.45 μs 1.79 μs 1.23 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSetReverse<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7894518972830449 > 1.5267697155497022. IsChangePoint: Marked as a change because one of 2/17/2022 3:48:53 PM, 2/22/2022 5:41:06 PM, 2/23/2022 9:24:01 AM, 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -195.83408055766438 (T) = (0 -1788.8803178898627) / Math.Sqrt((126.66793590438694 / (46)) + (1.116326386094395 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.22696089325172697 = (1457.9766378282202 - 1788.8803178898627) / 1457.9766378282202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEach(Int32).List(Size%3a%20512).html>) 284.33 ns 513.05 ns 1.80 0.01 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateForEach(Int32).ConcurrentQueue(Size%3a%20512).html>) 2.94 μs 3.28 μs 1.11 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 513.0496577748503 > 298.78029402071047. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -2463.031515644672 (T) = (0 -512.8628721075002) / Math.Sqrt((0.05548813997634782 / (46)) + (0.08103532892285802 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.8007315182353199 = (284.8080721162116 - 512.8628721075002) / 284.8080721162116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.ConcurrentQueue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2787198077499626 > 3.087562155516451. IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 3/31/2022 1:45:25 PM, 4/14/2022 8:50:18 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -332.3100898273043 (T) = (0 -3283.8440983883074) / Math.Sqrt((5.550079904278217 / (37)) + (18.319366785931713 / (20))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (20) - 2, .025) and -0.11666977225603364 = (2940.7477304179924 - 3283.8440983883074) / 2940.7477304179924 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.AddGivenSize(String).HashSet(Size%3a%20512).html>) 8.11 μs 8.88 μs 1.09 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<String>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.876139627833837 > 8.577192635751029. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/24/2022 10:17:16 AM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -23.196036177514152 (T) = (0 -8800.573426243192) / Math.Sqrt((4958.130518759598 / (46)) + (6837.403909346844 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.07663830165230878 = (8174.122556049713 - 8800.573426243192) / 8174.122556049713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.50).html>) 3.75 ns 4.77 ns 1.27 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'
### Payloads [Baseline]() [Compare]() ### Histogram #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.50) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.76838906128668 > 3.9371540223881687. IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 3/31/2022 1:45:25 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsRegressionStdDev: Marked as regression because -151.29650514674873 (T) = (0 -4.765355218038869) / Math.Sqrt((0.001989189951303827 / (46)) + (2.0627019955910152E-05 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.2710689558754346 = (3.7490926011616605 - 4.765355218038869) / 3.7490926011616605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)