search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Linux/x64: 37 Regressions on 10/2/2022 4:42:49 PM #8170

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSetReverse(Int32).IList(Size%3a%20512).html>) 1.02 μs 1.12 μs 1.10 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IndexerSetReverse<Int32>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.1191492010648483 > 1.058274772156936. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/23/2022 11:46:46 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -24.18677741225972 (T) = (0 -1119.209942980478) / Math.Sqrt((1285.8017795741712 / (50)) + (0.8269049011595291 / (17))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (17) - 2, .025) and -0.12320802546937858 = (996.4404790579823 - 1119.209942980478) / 996.4404790579823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayAssign2D - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Array.ArrayAssign2D.html>) 521.06 μs 553.68 μs 1.06 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Array.ArrayAssign2D ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 553.6768756157637 > 546.7450258380507. IsChangePoint: Marked as a change because one of 8/25/2022 4:20:58 PM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -24.118095312852606 (T) = (0 -555102.3271162942) / Math.Sqrt((51452841.24071671 / (48)) + (9471852.344113575 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.058711292313167615 = (524318.8876388167 - 555102.3271162942) / 524318.8876388167 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FixedTimeEquals_256Bit_CascadingErrors - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors.html>) 89.32 ns 101.87 ns 1.14 0.06 False
[FixedTimeEquals_256Bit_Equal - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal.html>) 89.41 ns 101.75 ns 1.14 0.05 False
[FixedTimeEquals_256Bit_VersusZero - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero.html>) 89.65 ns 101.94 ns 1.14 0.04 False
[FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent.html>) 89.35 ns 101.95 ns 1.14 0.07 False
[FixedTimeEquals_256Bit_SameReference - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference.html>) 89.32 ns 102.48 ns 1.15 0.06 False
[FixedTimeEquals_256Bit_AllBitsDifferent - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent.html>) 89.35 ns 101.89 ns 1.14 0.04 False
[FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent.html>) 92.02 ns 102.10 ns 1.11 0.06 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 101.87380635927364 > 93.82905184419745. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -28.74630033451175 (T) = (0 -101.87607090637835) / Math.Sqrt((7.444042096967231 / (45)) + (0.013254210339824428 / (17))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (17) - 2, .025) and -0.12998784165212512 = (90.1567850123308 - 101.87607090637835) / 90.1567850123308 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 101.74719143413583 > 93.89336143879461. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -45.79918534295584 (T) = (0 -101.8577879396858) / Math.Sqrt((3.3677183625209284 / (48)) + (0.016901197738990972 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.13628692574961482 = (89.64090462669863 - 101.8577879396858) / 89.64090462669863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 101.94139438038088 > 93.9150366126821. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -177.04429046629141 (T) = (0 -102.02016067143978) / Math.Sqrt((0.18165977130587918 / (48)) + (0.02233068859374652 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.14143368155095326 = (89.37896464805313 - 102.02016067143978) / 89.37896464805313 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 101.95402048244124 > 93.77577212482059. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -84.56736675909727 (T) = (0 -102.0296304966052) / Math.Sqrt((0.8918998022649848 / (48)) + (0.058319015257448835 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.14021364231327735 = (89.48290628202484 - 102.0296304966052) / 89.48290628202484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 102.47833577830026 > 93.75665681052554. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -49.72076699802116 (T) = (0 -102.74261964509095) / Math.Sqrt((2.168029985153037 / (46)) + (0.4079972285899113 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.1481944157804192 = (89.48190152558577 - 102.74261964509095) / 89.48190152558577 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 101.88709429809968 > 93.83802476385196. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -39.566968295218565 (T) = (0 -101.90955720786349) / Math.Sqrt((4.510422984836633 / (49)) + (0.06677195714101715 / (17))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (17) - 2, .025) and -0.13672836481770165 = (89.65163566073838 - 101.90955720786349) / 89.65163566073838 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 102.10140460099397 > 93.99497422708554. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -54.8892306538862 (T) = (0 -102.00854012065747) / Math.Sqrt((2.3713475660976178 / (48)) + (0.01774107614217332 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.13746959272862697 = (89.6802347708949 - 102.00854012065747) / 89.6802347708949 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEachNonGeneric(String).Queue(Size%3a%20512).html>) 3.48 μs 3.74 μs 1.08 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IterateForEachNonGeneric<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 3.7439070811329813 > 3.6522018972083927. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 1:34:50 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -14.482455988708583 (T) = (0 -3741.338686915732) / Math.Sqrt((12570.514849486908 / (49)) + (5.438764297444584 / (17))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (17) - 2, .025) and -0.06614233222626915 = (3509.230028511523 - 3741.338686915732) / 3509.230028511523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EmptyStaticFunction5Arg - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg.html>) 1.03 ms 1.24 ms 1.20 0.01 False
[GenericClassGenericStaticMethod - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod.html>) 108.82 μs 152.22 μs 1.40 0.03 False
[GenericClassGenericInstanceMethod - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod.html>) 108.81 μs 130.50 μs 1.20 0.03 False
[GenericGenericMethod - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.GenericGenericMethod.html>) 108.82 μs 152.19 μs 1.40 0.04 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.2395706638313608 > 1.0729234664152645. IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -117.64270431573617 (T) = (0 -1240725.8154501372) / Math.Sqrt((120840134.16947204 / (49)) + (12199375.412721153 / (17))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (17) - 2, .025) and -0.20363503053444165 = (1030815.6409333039 - 1240725.8154501372) / 1030815.6409333039 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 152.22090832743987 > 114.28374033854168. IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -1895.7215157484538 (T) = (0 -152275.93386958074) / Math.Sqrt((8405.278894814364 / (47)) + (5550.572578905553 / (16))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (16) - 2, .025) and -0.39948401747790957 = (108808.62658510808 - 152275.93386958074) / 108808.62658510808 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 130.50470095486114 > 114.24519019314236. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -461.2123886251916 (T) = (0 -130546.84088845899) / Math.Sqrt((58426.09055276158 / (31)) + (10631.777155692938 / (32))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .025) and -0.19953882353077898 = (108830.85926656486 - 130546.84088845899) / 108830.85926656486 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.GenericGenericMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 152.19131359223303 > 114.2611061469184. IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -975.8886282504294 (T) = (0 -152276.13611456723) / Math.Sqrt((41491.00130198657 / (48)) + (18952.31644532311 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.3988229848331971 = (108860.19014959596 - 152276.13611456723) / 108860.19014959596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayCopyToByteArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size%3a%20512).html>) 95.60 ns 126.14 ns 1.32 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 126.13600788542222 > 100.350848400725. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/15/2022 9:13:40 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -87.89667490645654 (T) = (0 -123.61344854605511) / Math.Sqrt((0.9707156197299862 / (46)) + (1.3129471948625266 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.2869632403790336 = (96.05048898649875 - 123.61344854605511) / 96.05048898649875 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ImmutableDictionary(Size%3a%20512).html>) 16.17 μs 17.19 μs 1.06 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 17.186324161907372 > 17.03297211532805. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/18/2022 3:00:51 AM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -20.04495529752394 (T) = (0 -17218.5499137577) / Math.Sqrt((131445.04653160425 / (50)) + (7335.744701781902 / (17))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (17) - 2, .025) and -0.0688350088372916 = (16109.642527978678 - 17218.5499137577) / 16109.642527978678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20JavaScript%2c%26Hello%2b(World)!%2c512).html>) 68.14 ns 75.46 ns 1.11 0.03 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20Url%2c%ef%bf%bd2020%2c16).html>) 34.66 ns 37.05 ns 1.07 0.01 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20UnsafeRelaxed%2chello%20%22there%22%2c512).html>) 55.77 ns 59.12 ns 1.06 0.03 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20Url%2c%26lorem%20ipsum%3ddolor%20sit%20amet%2c512).html>) 86.27 ns 98.28 ns 1.14 0.03 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20Url%2c%26lorem%20ipsum%3ddolor%20sit%20amet%2c16).html>) 54.55 ns 66.87 ns 1.23 0.03 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20UnsafeRelaxed%2chello%20%22there%22%2c16).html>) 24.53 ns 27.94 ns 1.14 0.03 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 75.45891344250703 > 71.59057095631795. IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -96.75248606180381 (T) = (0 -75.26637610866091) / Math.Sqrt((0.06696945702231177 / (46)) + (0.05934150018208765 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.09939521540552067 = (68.46161876454802 - 75.26637610866091) / 68.46161876454802 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,�2020,16) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 37.0520542219094 > 36.80138734905023. IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -45.43088467745857 (T) = (0 -37.03965372475892) / Math.Sqrt((0.10520207239747352 / (49)) + (0.0018443270844666934 / (17))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (17) - 2, .025) and -0.06185398597882691 = (34.88205931686118 - 37.03965372475892) / 34.88205931686118 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 59.11519402281862 > 58.80912249728358. IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -32.09321833271143 (T) = (0 -59.48437357860324) / Math.Sqrt((0.2823186715868247 / (48)) + (0.07812099708187155 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.05845206719154976 = (56.19940233707178 - 59.48437357860324) / 56.19940233707178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 98.2842556324795 > 90.47570981423918. IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -192.94249349295825 (T) = (0 -98.64690840380901) / Math.Sqrt((0.11441312745083426 / (47)) + (0.03165645758387946 / (17))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (17) - 2, .025) and -0.14705677337138653 = (86.00002257418322 - 98.64690840380901) / 86.00002257418322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 66.8706198983153 > 57.30091264399955. IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -349.36651303774437 (T) = (0 -66.77440281766287) / Math.Sqrt((0.013493546837205043 / (45)) + (0.015516148254800171 / (17))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (17) - 2, .025) and -0.22277812713079043 = (54.60876453060774 - 66.77440281766287) / 54.60876453060774 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 27.942568162728 > 25.98668843160833. IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -104.35013187549568 (T) = (0 -27.928411421261366) / Math.Sqrt((0.04373712143272482 / (48)) + (0.0008323703396157292 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.13093461827817454 = (24.694983219968822 - 27.928411421261366) / 24.694983219968822 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinkedList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(String).LinkedList(Size%3a%20512).html>) 1.03 ms 1.18 ms 1.15 0.05 False
[Span - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(String).Span(Size%3a%20512).html>) 654.30 μs 690.15 μs 1.05 0.02 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(String).ImmutableArray(Size%3a%20512).html>) 662.15 μs 724.38 μs 1.09 0.07 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.ContainsFalse<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.1833446181547618 > 1.0832607545703126. IsChangePoint: Marked as a change because one of 8/12/2022 3:52:12 PM, 8/13/2022 3:46:07 PM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -37.056420117555845 (T) = (0 -1186596.6350802893) / Math.Sqrt((529298832.99230963 / (47)) + (30016770.499953374 / (17))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (17) - 2, .025) and -0.1270358996687873 = (1052847.2388758918 - 1186596.6350802893) / 1052847.2388758918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 690.1475250000001 > 687.0074837890627. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/6/2022 4:25:34 AM, 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -240.45920667132526 (T) = (0 -690351.6285818799) / Math.Sqrt((568312.4135394027 / (32)) + (182939.7911598234 / (33))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (33) - 2, .025) and -0.05615771265906726 = (653644.4513043372 - 690351.6285818799) / 653644.4513043372 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 724.3780909090909 > 694.1622754497197. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -15.44247768408013 (T) = (0 -742396.503940108) / Math.Sqrt((128906656.94519809 / (32)) + (723554485.130265 / (34))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (34) - 2, .025) and -0.11687634744414535 = (664707.875351649 - 742396.503940108) / 664707.875351649 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetUInt32 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetUInt32.html>) 452.01 ns 499.79 ns 1.11 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.Json.Tests.Perf_Get.GetUInt32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 499.78711045854135 > 474.83119325977646. IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -104.45308448225059 (T) = (0 -499.50465871487273) / Math.Sqrt((0.37350216287160315 / (46)) + (3.3972301096823907 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.10541261243964438 = (451.8716840153167 - 499.50465871487273) / 451.8716840153167 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(String).Queue(Size%3a%20512).html>) 2.38 μs 2.97 μs 1.25 0.34 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(String).Dictionary(Size%3a%20512).html>) 2.25 μs 2.61 μs 1.16 0.28 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IterateForEach<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 2.9703725410663786 > 2.8824732778731668. IsChangePoint: Marked as a change because one of 9/1/2022 10:02:47 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -14.312401607601528 (T) = (0 -2959.9982252741984) / Math.Sqrt((29940.926513790895 / (48)) + (1266.1288472327824 / (17))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (17) - 2, .025) and -0.14648537524814234 = (2581.8019917031597 - 2959.9982252741984) / 2581.8019917031597 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 2.610394073166338 > 2.364217447985591. IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/2/2022 1:41:42 PM, 9/13/2022 12:23:36 PM, 9/30/2022 2:28:26 AM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -11.170872461089164 (T) = (0 -2542.80225701852) / Math.Sqrt((2777.339244105822 / (36)) + (13454.650733560013 / (28))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (28) - 2, .025) and -0.11575286873540368 = (2279.0013167526395 - 2542.80225701852) / 2279.0013167526395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSet(String).IList(Size%3a%20512).html>) 2.33 μs 2.57 μs 1.11 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IndexerSet<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 2.573107692624561 > 2.423149086766602. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/15/2022 5:23:15 PM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -45.41556944666433 (T) = (0 -2555.6108042812084) / Math.Sqrt((813.030623403252 / (47)) + (178.45823918814565 / (17))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (17) - 2, .025) and -0.10337745395297601 = (2316.170948686182 - 2555.6108042812084) / 2316.170948686182 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEachNonGeneric(Int32).Queue(Size%3a%20512).html>) 3.51 μs 3.85 μs 1.10 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IterateForEachNonGeneric<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 3.8533810372633908 > 3.6802205269573345. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 1:34:50 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -21.569172237717133 (T) = (0 -3826.2924051845243) / Math.Sqrt((4232.382427526157 / (47)) + (1823.1497527442732 / (16))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (16) - 2, .025) and -0.08756334853530458 = (3518.2248559016375 - 3826.2924051845243) / 3518.2248559016375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinkedList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrue(String).LinkedList(Size%3a%20512).html>) 518.53 μs 622.58 μs 1.20 0.06 False
[Span - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrue(String).Span(Size%3a%20512).html>) 327.39 μs 347.63 μs 1.06 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.ContainsTrue<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 622.580873076923 > 545.2606530141129. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -66.99427059695574 (T) = (0 -624745.4173164769) / Math.Sqrt((62099391.2663751 / (49)) + (19321263.780665934 / (17))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (17) - 2, .025) and -0.19941642025906742 = (520874.4909307939 - 624745.4173164769) / 520874.4909307939 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 347.63142341269844 > 344.3484277453926. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/6/2022 4:25:34 AM, 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -148.57015096839623 (T) = (0 -347589.4721965701) / Math.Sqrt((443346.5734776264 / (32)) + (84569.58354936335 / (31))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (31) - 2, .025) and -0.05824773230706416 = (328457.5639380747 - 347589.4721965701) / 328457.5639380747 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(Int32).Stack(Size%3a%20512).html>) 1.36 μs 1.59 μs 1.17 0.29 False
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(Int32).Queue(Size%3a%20512).html>) 1.24 μs 1.46 μs 1.18 0.09 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IterateForEach<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.5901002860309854 > 1.426471531536493. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -13.84931678432069 (T) = (0 -1603.0674050527593) / Math.Sqrt((5778.8316925278 / (50)) + (867.7528182367887 / (17))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (17) - 2, .025) and -0.12551401859639896 = (1424.2980350008484 - 1603.0674050527593) / 1424.2980350008484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.4621899095728097 > 1.2986668293543253. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -21.8070900290895 (T) = (0 -1453.6205946698833) / Math.Sqrt((122.38851264118112 / (49)) + (1535.5501875334464 / (17))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (17) - 2, .025) and -0.16895682624709568 = (1243.519488514125 - 1453.6205946698833) / 1243.519488514125 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchI.Fib.Test.html>) 102.32 μs 111.18 μs 1.09 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Fib*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.BenchI.Fib.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 111.18031028368796 > 107.48191859681373. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -54.46248985047572 (T) = (0 -111229.22629849812) / Math.Sqrt((1193798.272572826 / (50)) + (9044.86949508169 / (17))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (17) - 2, .025) and -0.08283360155614287 = (102720.51600416751 - 111229.22629849812) / 102720.51600416751 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Byte).SquareRootBenchmark.html>) 54.65 ns 67.81 ns 1.24 0.03 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Byte).MultiplyOperatorBenchmark.html>) 43.42 ns 46.16 ns 1.06 0.06 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Related Issues #### Regressions #### Improvements ### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_VectorOf<Byte>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 67.80843503845075 > 57.406870046068306. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -260.48683810074033 (T) = (0 -67.73842803346359) / Math.Sqrt((0.108193895598128 / (46)) + (0.004999748032966556 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.2465939553773044 = (54.338806747190844 - 67.73842803346359) / 54.338806747190844 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 46.15749840876673 > 45.53273343525582. IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/5/2022 12:22:07 PM falls between 9/26/2022 10:49:59 PM and 10/5/2022 12:22:07 PM. IsRegressionStdDev: Marked as regression because -14.384396062898068 (T) = (0 -46.02580291570069) / Math.Sqrt((0.8807585875100336 / (32)) + (0.04545368241382656 / (34))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (34) - 2, .025) and -0.05607067184429591 = (43.58212394566583 - 46.02580291570069) / 43.58212394566583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)