search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 6/7/2022 2:18:03 AM #6002

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.IndexerSet(String).IList(Size%3a%20512).html>) 3.37 μs 3.82 μs 1.14 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8226723338173607 > 3.565449668969648. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -80.07471252940552 (T) = (0 -3818.084615649107) / Math.Sqrt((250.31408318304096 / (28)) + (901.0149096155085 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.12510536017751572 = (3393.535175271675 - 3818.084615649107) / 3393.535175271675 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetBoolean - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Get.GetBoolean.html>) 111.67 ns 140.59 ns 1.26 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Get.GetBoolean ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 140.58773866340593 > 118.48963216227791. IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -25.725488593489946 (T) = (0 -141.08295936237707) / Math.Sqrt((30.2374142758382 / (28)) + (0.12399692471741908 / (46))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (46) - 2, .025) and -0.23414834804295429 = (114.3160460297166 - 141.08295936237707) / 114.3160460297166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.ContainsTrue(String).ImmutableArray(Size%3a%20512).html>) 501.42 μs 542.42 μs 1.08 0.02 False
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.ContainsTrue(String).Queue(Size%3a%20512).html>) 421.56 μs 498.64 μs 1.18 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 542.4178385416666 > 528.2143195564516. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -211.54451183494982 (T) = (0 -543124.3163784081) / Math.Sqrt((570526.169632209 / (27)) + (677754.0987426479 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.0796364198761652 = (503062.24056493456 - 543124.3163784081) / 503062.24056493456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 498.6400130208333 > 442.232546875. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -346.44404911641965 (T) = (0 -497891.70987714076) / Math.Sqrt((659621.9792570836 / (28)) + (1177354.685007564 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.18120860241459785 = (421510.3994835143 - 497891.70987714076) / 421510.3994835143 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Boolean.TryParse(value%3a%20%22%20True%20%22).html>) 13.40 ns 14.73 ns 1.10 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Boolean.TryParse(value: " True ") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.732207740314623 > 14.073653990582923. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/26/2022 5:29:00 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -210.25440029000634 (T) = (0 -14.735804248145062) / Math.Sqrt((0.00023503292049909433 / (27)) + (0.0014979705326432172 / (47))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (47) - 2, .025) and -0.09997517061354814 = (13.396488067931271 - 14.735804248145062) / 13.396488067931271 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent.html>) 127.66 ns 135.19 ns 1.06 0.06 False
[FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent.html>) 127.63 ns 139.72 ns 1.09 0.07 False
[FixedTimeEquals_256Bit_CascadingErrors - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors.html>) 127.65 ns 135.08 ns 1.06 0.02 False
[FixedTimeEquals_256Bit_Equal - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal.html>) 127.64 ns 135.12 ns 1.06 0.05 False
[FixedTimeEquals_256Bit_AllBitsDifferent - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent.html>) 127.65 ns 135.14 ns 1.06 0.03 False
[FixedTimeEquals_256Bit_VersusZero - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero.html>) 127.67 ns 135.13 ns 1.06 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.18515267652035 > 134.0460577752116. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/10/2022 11:18:06 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -19.22319557005147 (T) = (0 -135.6743735346639) / Math.Sqrt((1.8869589007000402 / (28)) + (4.247787018943805 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.059634246046424665 = (128.03887194178108 - 135.6743735346639) / 128.03887194178108 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 139.71608130969582 > 134.00928128598977. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -10.48876103158657 (T) = (0 -135.27456680685947) / Math.Sqrt((11.221775741587441 / (28)) + (1.891924904579313 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.05428776654394007 = (128.30895994392776 - 135.27456680685947) / 128.30895994392776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.08307495133315 > 134.02488189976927. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -64.6097960968064 (T) = (0 -135.14558606578098) / Math.Sqrt((0.052840923175883465 / (28)) + (0.5258319737225 / (46))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (46) - 2, .025) and -0.058394019582587865 = (127.68929488006748 - 135.14558606578098) / 127.68929488006748 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.12425115833574 > 134.04626987694732. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -34.44254654574429 (T) = (0 -135.14825380242172) / Math.Sqrt((0.9622378870737726 / (27)) + (0.43965256861622326 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.057283113438431274 = (127.82598348979666 - 135.14825380242172) / 127.82598348979666 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.13713117694317 > 134.04322239733904. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -132.42827974825056 (T) = (0 -135.07736941125083) / Math.Sqrt((0.02010050357944134 / (27)) + (0.10901602868738958 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.05787881607554112 = (127.68699718589053 - 135.07736941125083) / 127.68699718589053 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.1305683563748 > 134.04652717996183. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -32.89265642272345 (T) = (0 -135.1506283416059) / Math.Sqrt((1.2626573747139374 / (28)) + (0.14248000084715032 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.05640299583576128 = (127.9347264958133 - 135.1506283416059) / 127.9347264958133 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).IndexOfAnyTwoValues(Size%3a%20512).html>) 9.23 ns 10.89 ns 1.18 0.03 False
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).IndexOfAnyFourValues(Size%3a%20512).html>) 816.67 ns 890.27 ns 1.09 0.16 False
[LastIndexOfAnyValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).LastIndexOfAnyValues(Size%3a%20512).html>) 14.41 ns 16.19 ns 1.12 0.13 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.887879367142125 > 9.718898227851337. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -181.7076047515409 (T) = (0 -10.86476923739835) / Math.Sqrt((0.0012501793471902856 / (28)) + (0.0015326223914255898 / (46))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (46) - 2, .025) and -0.1732620382855848 = (9.260309191691197 - 10.86476923739835) / 9.260309191691197 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 890.2706332876086 > 857.296964070367. IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -25.99455188475274 (T) = (0 -890.3037901833422) / Math.Sqrt((204.1514995204707 / (27)) + (0.03749861533004516 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.08729936007173673 = (818.821221530566 - 890.3037901833422) / 818.821221530566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.19018351034084 > 15.150791167733137. IsChangePoint: Marked as a change because one of 4/21/2022 3:06:05 PM, 5/15/2022 3:27:46 PM, 5/23/2022 3:43:39 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -20.020364882524188 (T) = (0 -16.285318764226666) / Math.Sqrt((0.13822569114121927 / (24)) + (0.094966320283911 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.1202268914227283 = (14.537518148259883 - 16.285318764226666) / 14.537518148259883 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.ContainsFalse(String).ICollection(Size%3a%20512).html>) 992.88 μs 1.07 ms 1.08 0.02 False
[Span - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.ContainsFalse(String).Span(Size%3a%20512).html>) 878.94 μs 959.59 μs 1.09 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.069263013888889 > 1045.0923984375002. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -193.37821549308097 (T) = (0 -1069798.0502627336) / Math.Sqrt((1955563.2965726326 / (27)) + (3672854.286091947 / (46))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (46) - 2, .025) and -0.07589007793367812 = (994337.6857953327 - 1069798.0502627336) / 994337.6857953327 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 959.5855392156864 > 922.8814097222222. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -895.0964426468763 (T) = (0 -959306.4285714285) / Math.Sqrt((149330.9323191361 / (28)) + (128405.70500059312 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.09145975452980969 = (878920.5690728271 - 959306.4285714285) / 878920.5690728271 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Repeat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.Repeat.html>) 439.22 ns 496.47 ns 1.13 0.02 False
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20IEnumerable).html>) 653.04 ns 718.55 ns 1.10 0.02 False
[ElementAt - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.ElementAt(input%3a%20IList).html>) 7.61 ns 8.64 ns 1.14 0.06 False
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20Array).html>) 652.15 ns 717.90 ns 1.10 0.02 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Repeat ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 496.4693480853905 > 453.9888143651309. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -145.14896563978743 (T) = (0 -496.7534140711359) / Math.Sqrt((5.104321547775357 / (28)) + (0.3935630630542202 / (45))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (45) - 2, .025) and -0.14641306267276374 = (433.3110204737182 - 496.7534140711359) / 433.3110204737182 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 718.5480529688987 > 686.0029453342171. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/25/2022 1:42:32 AM, 5/26/2022 5:29:00 PM, 5/31/2022 6:06:22 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -39.74972567555694 (T) = (0 -714.263036530787) / Math.Sqrt((0.16601375759796672 / (24)) + (116.63400965434201 / (50))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (50) - 2, .025) and -0.09304301650626665 = (653.4628790857765 - 714.263036530787) / 653.4628790857765 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.638392051632794 > 7.987527544453885. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/15/2022 3:27:46 PM, 5/31/2022 6:06:22 PM, 6/7/2022 6:20:49 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -16.667582323722172 (T) = (0 -8.650520562256414) / Math.Sqrt((0.10007047566186958 / (32)) + (0.0002451677767418993 / (41))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (41) - 2, .025) and -0.12088868929815513 = (7.717555404785948 - 8.650520562256414) / 7.717555404785948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 717.8967892267262 > 684.4746305832492. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/31/2022 6:06:22 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -37.02959517585129 (T) = (0 -713.2650881325002) / Math.Sqrt((0.05631680399646645 / (24)) + (139.66336992252624 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.09403069862500087 = (651.9607621878853 - 713.2650881325002) / 651.9607621878853 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20Ordinal)).html>) 10.34 ns 11.41 ns 1.10 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.414967140095664 > 10.861675163255361. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/11/2022 7:02:26 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -131.63155134862916 (T) = (0 -11.248254869482274) / Math.Sqrt((0.0005773630658081794 / (28)) + (0.0012453884352921858 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.08734420623059701 = (10.34470483682038 - 11.248254869482274) / 10.34470483682038 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FooObjIsNull - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.CastingPerf2.CastingPerf.FooObjIsNull.html>) 374.20 μs 445.16 μs 1.19 0.27 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.CastingPerf2.CastingPerf.FooObjIsNull ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 445.16273148148144 > 392.9104862780449. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/28/2022 7:34:39 PM, 5/26/2022 5:29:00 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -842.1423678293157 (T) = (0 -445707.0241319167) / Math.Sqrt((90.0763060180124 / (28)) + (331445.6371712301 / (46))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (46) - 2, .025) and -0.1910726138151332 = (374206.42449688213 - 445707.0241319167) / 374206.42449688213 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.IndexerSetReverse(String).IList(Size%3a%20512).html>) 1.93 μs 2.43 μs 1.26 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSetReverse<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4267391419020967 > 2.023339808918929. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -239.33795643625731 (T) = (0 -2423.8492333321738) / Math.Sqrt((63.853462588622676 / (28)) + (91.40938882408769 / (46))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (46) - 2, .025) and -0.25625899733481994 = (1929.4184069323453 - 2423.8492333321738) / 1929.4184069323453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Collections.IterateForNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.IterateForNonGeneric(String).ArrayList(Size%3a%20512).html>) 1.45 μs 1.61 μs 1.11 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForNonGeneric<String>.ArrayList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6079523895603072 > 1.525246902650803. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/31/2022 2:22:18 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -77.08025300177809 (T) = (0 -1610.8615611087512) / Math.Sqrt((1.069924700459482 / (28)) + (196.88356851082494 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.10911629578212241 = (1452.38291713387 - 1610.8615611087512) / 1452.38291713387 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_VectorOf(Byte).MultiplyBenchmark.html>) 51.16 ns 59.20 ns 1.16 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.200736153508466 > 53.72414414261053. IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -205.82258748158094 (T) = (0 -59.17876516110923) / Math.Sqrt((0.026808829288095436 / (26)) + (0.023715877041414486 / (47))) is less than -1.993943367845047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (47) - 2, .025) and -0.15780300734234678 = (51.112982766342796 - 59.17876516110923) / 51.112982766342796 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 54cf23c8d0a26e3f2abdc5f87422e52ad93215c8
Compare bab1a1f239325746873d2ae795ea343e7901dbce
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_VectorOf(SByte).MultiplyBenchmark.html>) 51.28 ns 59.27 ns 1.16 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.26635708289895 > 53.92235699534439. IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -189.6804650053034 (T) = (0 -59.18507134271211) / Math.Sqrt((0.03672361959365323 / (28)) + (0.01916672828056394 / (46))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (46) - 2, .025) and -0.15371212859993075 = (51.299687223133574 - 59.18507134271211) / 51.299687223133574 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)