[Perf] Windows/x64: 5 Regressions on 10/27/2022 5:53:52 AM

[performanceautofiler[bot]]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	65233ebb1368ab5eebe76c05aac852b9b6a8cde8
Compare	060a1967649469fa71986dc92bb41629c0970946
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cs%5ba-zA-Z%5d%7b0%2c12%7ding%5c%5cs%22%2c%20Options%3a%20NonBacktracking).html>)	3.19 ms	3.36 ms	1.05	0.45	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bLl%7d%22%2c%20Options%3a%20NonBacktracking).html>)	41.44 ms	44.25 ms	1.07	0.05	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bL%7d%22%2c%20Options%3a%20NonBacktracking).html>)	43.59 ms	46.01 ms	1.06	0.01	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%5c%5cs%2bHolmes%22%2c%20Options%3a%20None).html>)	11.79 ms	12.73 ms	1.08	0.00	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)the%22%2c%20Options%3a%20NonBacktracking).html>)	1.77 ms	1.89 ms	1.07	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\s[a-zA-Z]{0,12}ing\\s", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 3.3619626447876434 > 3.351503794871795. IsChangePoint: Marked as a change because one of 10/27/2022 4:20:56 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM. IsRegressionStdDev: Marked as regression because -4.374781785428922 (T) = (0 -4334146.337784665) / Math.Sqrt((368215818552.9464 / (31)) + (341295668086.52264 / (20))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (20) - 2, .025) and -0.20732209715726888 = (3589884.048332868 - 4334146.337784665) / 3589884.048332868 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Ll}", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 44.253725 > 43.677789895833335. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM. IsRegressionStdDev: Marked as regression because -15.586370082598885 (T) = (0 -45241723.826821335) / Math.Sqrt((349138552703.5582 / (29)) + (654058300840.8567 / (21))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (21) - 2, .025) and -0.07711403963920942 = (42002724.0959328 - 45241723.826821335) / 42002724.0959328 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{L}", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 46.01152833333334 > 45.238524333333345. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM. IsRegressionStdDev: Marked as regression because -17.41719358966226 (T) = (0 -46330432.678571425) / Math.Sqrt((178359246754.05502 / (30)) + (496351029352.1286 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.06912706152553968 = (43334823.65740741 - 46330432.678571425) / 43334823.65740741 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 12.733612307692308 > 12.397918333333331. IsChangePoint: Marked as a change because one of 10/25/2022 10:16:52 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM. IsRegressionStdDev: Marked as regression because -12.569256689049226 (T) = (0 -12641845.35349707) / Math.Sqrt((37640729786.32659 / (23)) + (38357731840.29603 / (28))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (28) - 2, .025) and -0.05765988474595878 = (11952656.554175293 - 12641845.35349707) / 11952656.554175293 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.8887 > 1.8177371757515222. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM. IsRegressionStdDev: Marked as regression because -13.563955283741107 (T) = (0 -1860928.9201048084) / Math.Sqrt((489281841.48781526 / (30)) + (880381160.6672003 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.05889687170527084 = (1757422.2474639362 - 1860928.9201048084) / 1757422.2474639362 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

[cincuranet]

1st is noise. The rest seems to be valid regression.