search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Alpine/x64: 17 Regressions on 11/18/2022 3:21:53 PM #9893

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GenericClassGenericInstanceMethod - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod.html>) 108.73 μs 130.52 μs 1.20 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.5210793154762 > 114.17612705453726. IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -24.380313131713862 (T) = (0 -130485.7840658443) / Math.Sqrt((25576347.584813166 / (36)) + (1059.6472425941176 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.18693472250212076 = (109935.0971810593 - 130485.7840658443) / 109935.0971810593 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.IterateForEachNonGeneric(String).Stack(Size%3a%20512).html>) 2.81 μs 3.04 μs 1.08 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.044212637861684 > 2.8944304160522543. IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/2/2022 4:08:26 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -15.895175878003805 (T) = (0 -3026.6995640135547) / Math.Sqrt((5625.553957553032 / (36)) + (203.02582025592025 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.07308957380479392 = (2820.546986848409 - 3026.6995640135547) / 2820.546986848409 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).SumBenchmark.html>) 17.95 ns 20.91 ns 1.16 0.10 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GetHashCodeBenchmark.html>) 34.31 ns 37.62 ns 1.10 0.03 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.91297770216486 > 18.807275476424266. IsChangePoint: Marked as a change because one of 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -25.07637550854759 (T) = (0 -21.073730938615775) / Math.Sqrt((0.49979451920146106 / (36)) + (0.0284492530011052 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.17428269464497964 = (17.946045730484844 - 21.073730938615775) / 17.946045730484844 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.61614180798083 > 35.517002623760675. IsChangePoint: Marked as a change because one of 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -54.78089239243746 (T) = (0 -37.56788207050931) / Math.Sqrt((0.05512377316049429 / (36)) + (0.04830165594630149 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.10671149767867145 = (33.9455062582326 - 37.56788207050931) / 33.9455062582326 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.IterateForEach(Int32).ImmutableStack(Size%3a%20512).html>) 1.24 μs 1.46 μs 1.18 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.ImmutableStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4603181255257502 > 1.301538402217668. IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -14.708210765938766 (T) = (0 -1426.2013717459913) / Math.Sqrt((5.34564749365606 / (36)) + (2764.299666941192 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.1514986385464668 = (1238.5610577414845 - 1426.2013717459913) / 1238.5610577414845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BenchAssignJagged - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/ByteMark.BenchAssignJagged.html>) 730.89 ms 786.99 ms 1.08 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### ByteMark.BenchAssignJagged ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 786.9886851333333 > 766.7362458350001. IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/31/2022 4:38:04 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -18.09997311988483 (T) = (0 -777495114.5754015) / Math.Sqrt((145673345675285.66 / (36)) + (34615694582610.566 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.06091273386112152 = (732854917.9966571 - 777495114.5754015) / 732854917.9966571 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20None).html>) 1.19 μs 1.28 μs 1.08 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2819331875665096 > 1.2515426190376497. IsChangePoint: Marked as a change because one of 9/7/2022 8:07:21 PM, 10/17/2022 3:19:47 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -21.535011646150267 (T) = (0 -1281.6122191916552) / Math.Sqrt((534.2345983568636 / (36)) + (5.412276007664892 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.06999970676719355 = (1197.7687573988312 - 1281.6122191916552) / 1197.7687573988312 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ImmutableDictionary(Size%3a%20512).html>) 16.57 μs 17.64 μs 1.06 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.641653733521657 > 17.4200375144449. IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 10/19/2022 7:27:15 PM, 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -17.034442280263338 (T) = (0 -17512.284793067793) / Math.Sqrt((37155.69423554563 / (36)) + (30183.14933437769 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.054341633327669545 = (16609.68725838535 - 17512.284793067793) / 16609.68725838535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.Sort(IntStruct).List(Size%3a%20512).html>) 2.80 μs 3.08 μs 1.10 0.05 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.Sort(IntStruct).Array(Size%3a%20512).html>) 2.74 μs 3.08 μs 1.13 0.20 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntStruct>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0777761571428575 > 2.9412064500000006. IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/19/2022 7:27:15 PM, 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -21.54308140301448 (T) = (0 -3059.1170262917476) / Math.Sqrt((4326.955653840129 / (36)) + (203.47110603443545 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.08809075207388908 = (2811.453934757835 - 3059.1170262917476) / 2811.453934757835 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.084383533333333 > 2.8712405700000003. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 2:35:34 PM, 10/19/2022 1:55:59 PM, 11/18/2022 4:41:14 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -8.336988864973282 (T) = (0 -3067.7445135482285) / Math.Sqrt((14134.130696257833 / (38)) + (8983.404453824516 / (15))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (15) - 2, .025) and -0.0925103715864221 = (2807.977474020307 - 3067.7445135482285) / 2807.977474020307 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsFalse(String).ImmutableArray(Size%3a%20512).html>) 711.23 μs 764.32 μs 1.07 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 764.3244740646259 > 745.9562296875. IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -13.588450053062479 (T) = (0 -763854.8113745177) / Math.Sqrt((1397696034.1038363 / (36)) + (593970.1102218647 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.12472577001749327 = (679147.6035644113 - 763854.8113745177) / 679147.6035644113 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Numerics.Tests.Perf_VectorOf(Single).GetHashCodeBenchmark.html>) 22.01 ns 24.08 ns 1.09 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Single>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.079447353452792 > 23.384086252238674. IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -32.70267181300378 (T) = (0 -24.04950620448027) / Math.Sqrt((0.060814121703979575 / (36)) + (0.024849398863875608 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.08263912612244607 = (22.21377892614633 - 24.04950620448027) / 22.21377892614633 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsTrue(Int32).ImmutableSortedSet(Size%3a%20512).html>) 22.63 μs 23.90 μs 1.06 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.898483521279303 > 23.76255355191751. IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -46.12243879425553 (T) = (0 -23941.47887572976) / Math.Sqrt((23080.789841016787 / (36)) + (2131.789062327697 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.05634184082640178 = (22664.518198957037 - 23941.47887572976) / 22664.518198957037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsFalse(Int32).SortedSet(Size%3a%20512).html>) 25.95 μs 27.35 μs 1.05 0.00 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsFalse(Int32).ImmutableSortedSet(Size%3a%20512).html>) 26.73 μs 28.67 μs 1.07 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.34623009705658 > 27.238820185211477. IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -34.362326323540024 (T) = (0 -27341.51871747732) / Math.Sqrt((7718.7058188871815 / (36)) + (24967.0866469771 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.05436254450988515 = (25931.80008133434 - 27341.51871747732) / 25931.80008133434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.672668837759463 > 28.04679612174701. IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -92.70735372408214 (T) = (0 -28656.4252738185) / Math.Sqrt((3252.812461450299 / (36)) + (6217.325724904317 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.07421732313171099 = (26676.56223442312 - 28656.4252738185) / 26676.56223442312 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GetHashCodeBenchmark.html>) 17.76 ns 19.47 ns 1.10 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.47150777366021 > 18.76603778928385. IsChangePoint: Marked as a change because one of 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -28.296579518746444 (T) = (0 -19.542356319794916) / Math.Sqrt((0.04774913107892369 / (36)) + (0.04050357056374244 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.09671039543810414 = (17.819067277089417 - 19.542356319794916) / 17.819067277089417 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 2725a849bd66b03ec3c295f48d56884c6ba5e3b4
Compare d15a19fd4a9dd21fb768b6de4870a2d80acb3669
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GetHashCodeBenchmark.html>) 18.12 ns 19.41 ns 1.07 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.410615039986183 > 18.92652262057172. IsChangePoint: Marked as a change because one of 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsRegressionStdDev: Marked as regression because -26.3123734336565 (T) = (0 -19.51673641951539) / Math.Sqrt((0.05088240914166762 / (36)) + (0.04114053479914001 / (17))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (17) - 2, .025) and -0.09107548458340443 = (17.887613364319417 - 19.51673641951539) / 17.887613364319417 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)