dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 19 Regressions on 10/25/2022 7:09:53 PM #9339

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20None).html>) 290.86 ns 314.18 ns 1.08 0.24 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20None).html>) 288.43 ns 329.30 ns 1.14 0.12 False
[IP_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 105.58 ns 127.98 ns 1.21 0.06 False
[OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20Compiled).html>) 73.39 ns 88.48 ns 1.21 0.08 False
[MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20None).html>) 1.15 μs 1.32 μs 1.15 0.09 False
[IP_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options%3a%20None).html>) 623.64 ns 694.32 ns 1.11 0.04 False
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20Compiled).html>) 114.47 ns 137.96 ns 1.21 0.06 False
[IP_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options%3a%20None).html>) 617.02 ns 693.57 ns 1.12 0.03 False
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) 123.89 ns 135.01 ns 1.09 0.10 False
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20Compiled).html>) 107.89 ns 116.53 ns 1.08 0.09 False
[IP_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 105.63 ns 129.41 ns 1.23 0.07 False
[OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) 72.12 ns 79.30 ns 1.10 0.12 False

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 314.18155598013976 > 305.709328866613. IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -14.262714242293352 (T) = (0 -319.81617684574087) / Math.Sqrt((59.920830731501 / (21)) + (16.21534166480297 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.08977971939113513 = (293.46864430953406 - 319.81617684574087) / 293.46864430953406 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 329.29526045065717 > 304.4809632118888. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -12.719152402962218 (T) = (0 -318.74306318622223) / Math.Sqrt((31.157663241228725 / (21)) + (102.06962218591627 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.09800587693032031 = (290.2926750058276 - 318.74306318622223) / 290.2926750058276 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 127.98152446675034 > 111.00532349382794. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -12.554321638732171 (T) = (0 -130.76991711995652) / Math.Sqrt((36.98744598474446 / (30)) + (47.477261079700604 / (20))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (20) - 2, .025) and -0.22297908789780138 = (106.92735338977795 - 130.76991711995652) / 106.92735338977795 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 88.48138701399854 > 78.09999538957418. IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 10/27/2022 12:34:03 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -20.85854820331859 (T) = (0 -86.47125217014877) / Math.Sqrt((2.7830911020001428 / (30)) + (3.70828897467086 / (20))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (20) - 2, .025) and -0.14577300548839106 = (75.46979354195031 - 86.47125217014877) / 75.46979354195031 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.3212636122454648 > 1.2113325437083777. IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/28/2022 11:19:50 AM, 10/26/2022 4:56:52 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -9.593306907666163 (T) = (0 -1233.0164638371373) / Math.Sqrt((883.969054626842 / (28)) + (879.245781851858 / (22))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (22) - 2, .025) and -0.07044084987322782 = (1151.8772513055376 - 1233.0164638371373) / 1151.8772513055376 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 694.3243433144071 > 649.9391753240619. IsChangePoint: Marked as a change because one of 8/30/2022 3:47:49 PM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -29.89926935253691 (T) = (0 -708.620274510438) / Math.Sqrt((75.79268832683788 / (25)) + (125.38848848702766 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.1359680205064316 = (623.8030135694528 - 708.620274510438) / 623.8030135694528 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 137.96198602198197 > 118.64457484827508. IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 10/4/2022 2:54:32 AM, 10/4/2022 3:13:24 PM, 10/17/2022 3:19:47 PM, 10/28/2022 6:19:52 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -6.978988948733122 (T) = (0 -131.75530556561972) / Math.Sqrt((28.477824416785587 / (40)) + (45.912961304664435 / (10))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (10) - 2, .025) and -0.13892858167622077 = (115.68355354793937 - 131.75530556561972) / 115.68355354793937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 693.5703529359273 > 650.146447853521. IsChangePoint: Marked as a change because one of 8/30/2022 3:47:49 PM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -24.818011839903317 (T) = (0 -708.5135518613797) / Math.Sqrt((147.5822436663469 / (25)) + (139.19993961742185 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.13460795596566116 = (624.4567104752638 - 708.5135518613797) / 624.4567104752638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 135.0050327161922 > 130.43276501654353. IsChangePoint: Marked as a change because one of 8/17/2022 11:40:52 AM, 9/13/2022 12:23:36 PM, 9/22/2022 3:16:12 AM, 10/1/2022 2:31:50 PM, 10/11/2022 2:16:14 AM, 10/26/2022 2:14:52 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -12.493201343636592 (T) = (0 -137.91592255142385) / Math.Sqrt((4.155644390535771 / (27)) + (29.56710702605403 / (23))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .025) and -0.12193351924688246 = (122.92700074065202 - 137.91592255142385) / 122.92700074065202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 116.53318210201022 > 113.17838674501087. IsChangePoint: Marked as a change because one of 8/30/2022 3:47:49 PM, 9/9/2022 3:12:19 AM, 9/23/2022 1:16:35 AM, 10/26/2022 2:14:52 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -7.454372008438738 (T) = (0 -113.58125817153014) / Math.Sqrt((0.9376591043423428 / (27)) + (17.329139818328414 / (23))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .025) and -0.0618708003430407 = (106.96335009385074 - 113.58125817153014) / 106.96335009385074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 129.411334010496 > 110.91068039684593. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -16.425808541896384 (T) = (0 -131.58655815731882) / Math.Sqrt((2.499156880497458 / (30)) + (48.65248848957509 / (20))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (20) - 2, .025) and -0.2468822866894686 = (105.53246249626928 - 131.58655815731882) / 105.53246249626928 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 79.29544915446452 > 76.024892831368. IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 9/20/2022 6:55:31 AM, 9/29/2022 7:42:19 AM, 10/28/2022 6:19:52 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -10.994381836285356 (T) = (0 -81.31403620422863) / Math.Sqrt((6.233183332053246 / (40)) + (1.346169920368003 / (10))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (10) - 2, .025) and -0.07859537315147344 = (75.38882349053914 - 81.31403620422863) / 75.38882349053914 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Max - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.MathBenchmarks.Single.Max.html>) 5.58 μs 6.97 μs 1.25 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.MathBenchmarks.Single.Max ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 6.9692840807675145 > 5.858076227678572. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -2599.470724192556 (T) = (0 -6969.44518861027) / Math.Sqrt((2.8262387411049192 / (21)) + (4.410878814666537 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.2495378314093305 = (5577.618390913032 - 6969.44518861027) / 5577.618390913032 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Interpol_Scalar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/BilinearTest.Interpol_Scalar.html>) 7.33 μs 8.00 μs 1.09 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BilinearTest*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### BilinearTest.Interpol_Scalar ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 7.998651270897306 > 7.695273448586106. IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -26.813197195124012 (T) = (0 -8426.272653975375) / Math.Sqrt((108.00942095759453 / (21)) + (48332.954833146905 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.14956833040968306 = (7329.945015945612 - 8426.272653975375) / 7329.945015945612 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Load - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName%3a%20%22deep.xml%22).html>) 62.79 μs 66.59 μs 1.06 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "deep.xml") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 66.59070434935727 > 65.74216164314085. IsChangePoint: Marked as a change because one of 9/8/2022 12:20:47 PM, 10/3/2022 8:45:37 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -11.805715912107944 (T) = (0 -69107.0664009035) / Math.Sqrt((137672.30600986793 / (21)) + (8128076.482795787 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.1007063618996137 = (62784.289064739845 - 69107.0664009035) / 62784.289064739845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetByteCount - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Perf_Utf8Encoding.GetByteCount(Input%3a%20EnglishAllAscii).html>) 4.55 μs 5.87 μs 1.29 0.01 False
[GetBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Perf_Utf8Encoding.GetBytes(Input%3a%20EnglishAllAscii).html>) 17.50 μs 18.89 μs 1.08 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 5.865323803262854 > 4.776184749337968. IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/17/2022 2:11:12 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -2626.82983211771 (T) = (0 -5864.837950970149) / Math.Sqrt((2.898675263608412 / (21)) + (3.277001959633702 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.2893396891329439 = (4548.714353867552 - 5864.837950970149) / 4548.714353867552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 18.893699978485365 > 18.42171688822383. IsChangePoint: Marked as a change because one of 8/25/2022 6:55:57 AM, 9/13/2022 12:23:36 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -46.20578070452493 (T) = (0 -18746.473231888493) / Math.Sqrt((7245.809564469178 / (21)) + (9593.540119182004 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.06846393431584315 = (17545.255978988378 - 18746.473231888493) / 17545.255978988378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.MathBenchmarks.Double.Min.html>) 4.99 μs 5.30 μs 1.06 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.MathBenchmarks.Double.Min ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 5.304080220713072 > 5.236473993355747. IsChangePoint: Marked as a change because one of 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -42.6408613283043 (T) = (0 -5305.156167487145) / Math.Sqrt((1103.426327925638 / (21)) + (1.9683360087640687 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.06190955257267823 = (4995.8644355674105 - 5305.156167487145) / 4995.8644355674105 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22%5b%5c%5cw%5d%2b%3a%2f%2f%5b%5e%2f%5c%5cs%3f%23%5d%2b%5b%5e%5c%5cs%3f%23%5d%2b(%3f%3a%5c%5c%3f%5b%5e%5c%5cs%23%5d)%3f(%3f%3a%23%5b%5e%5c%5cs%5d)%3f%22%2c%20Options%3a%20None).html>) 2.48 ms 2.65 ms 1.07 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w]+://[^/\\s?#]+[^\\s?#]+(?:\\?[^\\s#]*)?(?:#[^\\s]*)?", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 2.64575375 > 2.5895136450892857. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM. IsRegressionStdDev: Marked as regression because -14.300301358778695 (T) = (0 -2632152.5057904203) / Math.Sqrt((592141683.4922543 / (21)) + (2397728320.8557887 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.060679211749123205 = (2481572.634434726 - 2632152.5057904203) / 2481572.634434726 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
AndyAyersMS commented 1 year ago

Duping to https://github.com/dotnet/runtime/issues/77732