search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/6/2022 2:58:43 AM #4598

Closed performanceautofiler[bot] closed 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 0c5e7daf58e5d79cb7a2a8767d1f0e6ec4de4dd4
Compare 22870c2e31ae3c028cece52babd3cae57b560713
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%5cs%2b%5cw%2b%22%2c%20Options%3a%20None).html>) 1.34 ms 1.21 ms 0.90 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cw%2b%5cs%2bHolmes%22%2c%20Options%3a%20None).html>) 14.53 ms 13.26 ms 0.91 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20None).html>) 37.73 ms 34.34 ms 0.91 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20None).html>) 6.43 ms 5.82 ms 0.91 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-zA-Z%5d%2bing%22%2c%20Options%3a%20None).html>) 21.08 ms 18.42 ms 0.87 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cs%5ba-zA-Z%5d%7b0%2c12%7ding%5cs%22%2c%20Options%3a%20Compiled).html>) 8.06 ms 7.44 ms 0.92 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-zA-Z%5d%2bing%22%2c%20Options%3a%20Compiled).html>) 6.82 ms 5.39 ms 0.79 0.02 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cs%5ba-zA-Z%5d%7b0%2c12%7ding%5cs%22%2c%20Options%3a%20None).html>) 23.42 ms 21.95 ms 0.94 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cw%2b%5cs%2bHolmes%5cs%2b%5cw%2b%22%2c%20Options%3a%20None).html>) 14.25 ms 13.22 ms 0.93 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20NonBacktracking).html>) 5.97 ms 5.46 ms 0.91 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fs).*%22%2c%20Options%3a%20None).html>) 2.80 ms 1.96 ms 0.70 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.208325232255667 < 1.2478167628968255. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 3/2/2022 5:45:31 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 35.1378185018227 (T) = (0 -1215558.9081178172) / Math.Sqrt((1695447738.713635 / (299)) + (56975045.11408269 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.07265560953592029 = (1310795.55838959 - 1215558.9081178172) / 1310795.55838959 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.256509663865545 < 13.664556845238097. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/4/2022 7:13:54 PM, 3/15/2022 12:53:39 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 44.91702646427934 (T) = (0 -13287229.62595219) / Math.Sqrt((130897936308.39133 / (299)) + (10327901153.26862 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.08429496925566858 = (14510381.814929701 - 13287229.62595219) / 14510381.814929701 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.34144666666667 < 35.7888500103022. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 41.769101854117224 (T) = (0 -33417490.255448334) / Math.Sqrt((5665159179370.678 / (299)) + (172864554011.7368 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.16231242379953628 = (39892546.10534098 - 33417490.255448334) / 39892546.10534098 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.818703333333333 < 6.298793213541667. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 34.066946102376555 (T) = (0 -5875688.294123566) / Math.Sqrt((175058889475.20807 / (299)) + (11483975620.052769 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.149783073191855 = (6910810.769413721 - 5875688.294123566) / 6910810.769413721 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.42309642857143 < 19.103080629629627. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/17/2022 11:24:40 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 26.918479129220863 (T) = (0 -18708527.3430907) / Math.Sqrt((698464240627.8112 / (299)) + (129875573071.71802 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10143863543270906 = (20820533.889858413 - 18708527.3430907) / 20820533.889858413 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.443959803921568 < 7.656950756630824. IsChangePoint: Marked as a change because one of 2/15/2022 12:48:49 AM, 2/26/2022 10:27:55 AM, 4/6/2022 2:58:43 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 47.9253895692767 (T) = (0 -7450754.87320958) / Math.Sqrt((79506257920.35838 / (299)) + (486667932.2186225 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09727617070176513 = (8253637.083006543 - 7450754.87320958) / 8253637.083006543 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.390541666666667 < 6.472080899305554. IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 4/6/2022 2:58:43 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 111.5268703323342 (T) = (0 -5396841.534645909) / Math.Sqrt((49697531392.92658 / (299)) + (133512702.40520906 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.21237889795957493 = (6852078.392344691 - 5396841.534645909) / 6852078.392344691 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.954655 < 22.22261533333333. IsChangePoint: Marked as a change because one of 3/15/2022 11:00:34 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 33.28857005630521 (T) = (0 -22112334.221679054) / Math.Sqrt((692463204450.9968 / (299)) + (114994622656.2243 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10204276099933865 = (24625152.80381048 - 22112334.221679054) / 24625152.80381048 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.221574660633486 < 13.644914948979592. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/4/2022 7:13:54 PM, 3/14/2022 9:50:33 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 47.6476044539182 (T) = (0 -13278294.012406997) / Math.Sqrt((146096097540.4537 / (299)) + (7840552043.656682 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.08778622045781659 = (14556120.846005013 - 13278294.012406997) / 14556120.846005013 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.459245777777778 < 5.683609581985233. IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 123.3952895667946 (T) = (0 -5428199.26701829) / Math.Sqrt((3521013548.2321296 / (299)) + (233287367.91629964 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.08936722130463227 = (5960909.154615634 - 5428199.26701829) / 5960909.154615634 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.9561523359580049 < 2.663675920545746. IsChangePoint: Marked as a change because one of 2/22/2022 5:41:06 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 35.3942190132051 (T) = (0 -1956889.9417370015) / Math.Sqrt((83359503699.58563 / (279)) + (3906589.789152981 / (34))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (279) + (34) - 2, .975) and 0.23821006706813053 = (2568805.200937219 - 1956889.9417370015) / 2568805.200937219 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 0c5e7daf58e5d79cb7a2a8767d1f0e6ec4de4dd4
Compare 22870c2e31ae3c028cece52babd3cae57b560713
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MatchesSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options%3a%20None).html>) 183.99 μs 159.27 μs 0.87 0.01 False
[IP_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 207.44 ns 180.01 ns 0.87 0.07 False
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20None).html>) 445.55 ns 392.79 ns 0.88 0.03 False
[MatchesBoundary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options%3a%20IgnoreCase%2c%20Compiled).html>) 68.31 μs 58.99 μs 0.86 0.02 False
[Date_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 161.79 ns 137.22 ns 0.85 0.04 False
[OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) 283.92 ns 114.51 ns 0.40 0.03 True
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20None).html>) 436.03 ns 368.08 ns 0.84 0.02 False
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 269.92 ns 189.88 ns 0.70 0.01 True
[Email_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options%3a%20None).html>) 716.59 ns 669.81 ns 0.93 0.03 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 159.81 ns 131.68 ns 0.82 0.04 False
[MatchesBoundary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options%3a%20None).html>) 156.24 μs 132.40 μs 0.85 0.01 False
[MatchesSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options%3a%20IgnoreCase%2c%20Compiled).html>) 68.25 μs 59.51 μs 0.87 0.03 False
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 234.76 ns 137.35 ns 0.59 0.05 True
[Email_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 363.56 ns 216.17 ns 0.59 0.01 True
[IP_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 207.24 ns 176.02 ns 0.85 0.07 False
[Date_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options%3a%20None).html>) 434.97 ns 407.30 ns 0.94 0.02 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20None).html>) 419.67 ns 364.76 ns 0.87 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 159.26755952380952 < 174.40976853462644. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 65.9533956974435 (T) = (0 -159693.69278574912) / Math.Sqrt((63032784.7776563 / (299)) + (941187.4406202121 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.1678398284429754 = (191902.5906838969 - 159693.69278574912) / 191902.5906838969 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 180.00555063008179 < 197.40990950432297. IsChangePoint: Marked as a change because one of 2/25/2022 8:42:58 PM, 2/28/2022 8:13:22 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 41.89133435123132 (T) = (0 -176.83519540475834) / Math.Sqrt((313.6175106343846 / (299)) + (5.617236442027752 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.20699459792486163 = (222.99368319814164 - 176.83519540475834) / 222.99368319814164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 392.7886362517686 < 422.6015023993761. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 6:54:53 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 32.55025081823353 (T) = (0 -395.4064628213159) / Math.Sqrt((906.5309645156283 / (299)) + (9.967714841516223 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.13051786255700507 = (454.7608809815712 - 395.4064628213159) / 454.7608809815712 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 58.990732816711585 < 64.78615176953933. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 65.73269894820962 (T) = (0 -58811.331332171525) / Math.Sqrt((14044299.720251976 / (299)) + (54069.41164096688 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.19762520816936952 = (73296.58400408189 - 58811.331332171525) / 73296.58400408189 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 137.21519921153427 < 152.99997713316364. IsChangePoint: Marked as a change because one of 3/2/2022 5:45:31 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 49.459738620608896 (T) = (0 -137.38200935197094) / Math.Sqrt((119.6693979272007 / (299)) + (4.425044325300512 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.20772609750868173 = (173.40216422624923 - 137.38200935197094) / 173.40216422624923 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 114.50632820589594 < 270.2747071429732. IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 303.40054321195936 (T) = (0 -113.88245879864499) / Math.Sqrt((89.70782425500632 / (299)) + (1.4136539255812586 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.6089345323222289 = (291.21072610911665 - 113.88245879864499) / 291.21072610911665 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 368.0810361234425 < 413.910109377621. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 6:54:53 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 57.69587081357545 (T) = (0 -367.92471058086807) / Math.Sqrt((324.5614006112065 / (299)) + (27.18701395568304 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.17716120243725786 = (447.14069349994827 - 367.92471058086807) / 447.14069349994827 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 189.8768208556588 < 258.0564494365122. IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 123.25392198565255 (T) = (0 -189.6841046843345) / Math.Sqrt((44.95312241439831 / (299)) + (13.3634611005569 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.32386206415707824 = (280.5405445086597 - 189.6841046843345) / 280.5405445086597 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 669.8147106360559 < 689.7067662314303. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 2:30:39 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 29.081890118014712 (T) = (0 -676.5527630976767) / Math.Sqrt((832.6915697060651 / (299)) + (114.96329649186295 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.09644589548967361 = (748.768402158196 - 676.5527630976767) / 748.768402158196 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 131.68138840862537 < 150.12845930037815. IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 40.13175974128229 (T) = (0 -134.9891943073526) / Math.Sqrt((24.472026005627725 / (299)) + (13.42197453844389 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.17029261653716468 = (162.69494161178463 - 134.9891943073526) / 162.69494161178463 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 132.40159914712154 < 147.99084567338102. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 52.67766726704771 (T) = (0 -134495.24763810626) / Math.Sqrt((48084151.61595605 / (299)) + (3711135.3959401655 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.16909371446089969 = (161865.72418433975 - 134495.24763810626) / 161865.72418433975 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 59.51023130544994 < 65.35123190788394. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 48.26421510633015 (T) = (0 -59151.48206283086) / Math.Sqrt((31455726.02027454 / (299)) + (84737.42799947855 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2112119418744745 = (74990.33669880644 - 59151.48206283086) / 74990.33669880644 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 137.35323435384498 < 223.6998320364376. IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 102.53929360297066 (T) = (0 -141.03304327752053) / Math.Sqrt((79.07168532130618 / (299)) + (23.38285046039634 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.4150211368843219 = (241.090836216473 - 141.03304327752053) / 241.090836216473 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 216.16871600762903 < 345.1586794202808. IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 126.75744355139895 (T) = (0 -217.3983987162512) / Math.Sqrt((404.61232494570794 / (299)) + (16.64655403287898 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.44181309711565053 = (389.4724107514466 - 217.3983987162512) / 389.4724107514466 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 176.01981101081262 < 197.457768321945. IsChangePoint: Marked as a change because one of 2/25/2022 8:42:58 PM, 2/28/2022 8:13:22 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 42.35181825422404 (T) = (0 -176.91641814191055) / Math.Sqrt((307.4631255017377 / (299)) + (5.690658414539333 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.20745912555421803 = (223.22686923324542 - 176.91641814191055) / 223.22686923324542 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 407.299000867511 < 412.0002754088482. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/4/2022 7:13:54 PM, 3/15/2022 12:53:39 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 17.38412687786429 (T) = (0 -410.80369505234506) / Math.Sqrt((486.19258110200997 / (299)) + (186.80190259177468 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10146173930496136 = (457.1910991687539 - 410.80369505234506) / 457.1910991687539 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 364.76461747003447 < 398.8479123283007. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 6:54:53 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 43.41938259591461 (T) = (0 -363.4076244908247) / Math.Sqrt((113.77731804490293 / (299)) + (41.350154541040574 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.13117066535136818 = (418.2727378073536 - 363.4076244908247) / 418.2727378073536 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 0c5e7daf58e5d79cb7a2a8767d1f0e6ec4de4dd4
Compare 22870c2e31ae3c028cece52babd3cae57b560713
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22%5b%5cw%5d%2b%3a%2f%2f%5b%5e%2f%5cs%3f%23%5d%2b%5b%5e%5cs%3f%23%5d%2b(%3f%3a%5c%3f%5b%5e%5cs%23%5d)%3f(%3f%3a%23%5b%5e%5cs%5d)%3f%22%2c%20Options%3a%20None).html>) 3.97 ms 3.40 ms 0.86 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.3976007305936067 < 3.7958719735449735. IsChangePoint: Marked as a change because one of 2/8/2022 3:28:17 PM, 3/4/2022 7:13:54 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 45.2472431839781 (T) = (0 -3452920.8885089452) / Math.Sqrt((19271109188.072483 / (299)) + (1632770233.215582 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.12201659096136146 = (3932786.033268868 - 3452920.8885089452) / 3932786.033268868 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 0c5e7daf58e5d79cb7a2a8767d1f0e6ec4de4dd4
Compare 22870c2e31ae3c028cece52babd3cae57b560713
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2013%2c%20Options%3a%20None).html>) 146.79 ns 127.12 ns 0.87 0.04 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%205%2c%20Options%3a%20None).html>) 404.69 ns 374.71 ns 0.93 0.05 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2011%2c%20Options%3a%20None).html>) 125.75 ns 114.74 ns 0.91 0.06 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%208%2c%20Options%3a%20None).html>) 148.32 ns 137.46 ns 0.93 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%206%2c%20Options%3a%20None).html>) 149.31 ns 139.42 ns 0.93 0.05 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%203%2c%20Options%3a%20None).html>) 502.32 ns 466.53 ns 0.93 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%207%2c%20Options%3a%20None).html>) 154.81 ns 137.86 ns 0.89 0.05 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2012%2c%20Options%3a%20None).html>) 161.36 ns 127.08 ns 0.79 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%204%2c%20Options%3a%20None).html>) 397.79 ns 367.33 ns 0.92 0.05 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 127.1194818193246 < 139.47065871972464. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 6:54:53 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 41.37085562474911 (T) = (0 -127.9951724026922) / Math.Sqrt((46.39954868883245 / (299)) + (2.1211727915626426 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.13101278493872795 = (147.292353885399 - 127.9951724026922) / 147.292353885399 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 374.70781140591413 < 384.9060553402124. IsChangePoint: Marked as a change because one of 3/10/2022 1:12:51 AM, 3/15/2022 12:53:39 AM, 3/21/2022 1:28:14 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 29.314002753738386 (T) = (0 -377.17612433418157) / Math.Sqrt((344.82315253355733 / (279)) + (18.656827133067303 / (34))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (279) + (34) - 2, .975) and 0.09406050772507146 = (416.3369933096135 - 377.17612433418157) / 416.3369933096135 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 114.743840892665 < 119.12811025193753. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/18/2022 1:59:24 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 36.09296916205172 (T) = (0 -114.53767256303087) / Math.Sqrt((27.62661925791933 / (299)) + (0.8768485312800477 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.09774352232475093 = (126.94580243762643 - 114.53767256303087) / 126.94580243762643 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 137.46162249189737 < 140.9671811810452. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/16/2022 5:02:20 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 14.040500263280192 (T) = (0 -139.20136863634167) / Math.Sqrt((34.342060280885 / (299)) + (33.676631848251304 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.0958772381923159 = (153.96290693757712 - 139.20136863634167) / 153.96290693757712 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 139.42388132286666 < 142.12025707927143. IsChangePoint: Marked as a change because one of 3/16/2022 5:02:20 PM, 4/6/2022 2:58:43 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 34.879344979868286 (T) = (0 -140.39993121866019) / Math.Sqrt((25.43913757304133 / (299)) + (4.2181101019107565 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10283986852598896 = (156.49372535980473 - 140.39993121866019) / 156.49372535980473 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 466.5301336774781 < 479.8148313205662. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 6:54:53 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 20.91919332253568 (T) = (0 -462.88388427412826) / Math.Sqrt((550.8784678181162 / (299)) + (32.36498070550324 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.07023952467202521 = (497.8528304409209 - 462.88388427412826) / 497.8528304409209 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 137.85511159277354 < 141.70037590599037. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/16/2022 12:48:00 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 17.309054175427605 (T) = (0 -139.37935972083784) / Math.Sqrt((21.841183439121494 / (299)) + (21.325273818205883 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.09413839895624623 = (153.86385686316967 - 139.37935972083784) / 153.86385686316967 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 127.07534570824318 < 140.08192480066378. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/17/2022 6:54:53 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 40.3686290934696 (T) = (0 -128.06583679238227) / Math.Sqrt((50.336012367284965 / (299)) + (2.0900498515273958 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.13127627061386035 = (147.41837072054722 - 128.06583679238227) / 147.41837072054722 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 367.32521930786095 < 376.4287028989797. IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/15/2022 12:53:39 AM, 3/21/2022 1:28:14 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM. IsImprovementStdDev: Marked as improvement because 27.634826955599223 (T) = (0 -370.23219910528434) / Math.Sqrt((311.4882928675375 / (299)) + (27.485276521078994 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.0921703542080374 = (407.8212259551242 - 370.23219910528434) / 407.8212259551242 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
DrewScoggins commented 2 years ago

https://github.com/dotnet/runtime/pull/67184