dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 49 Improvements on 1/13/2023 3:30:06 PM #11912

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fm)%5eSherlock%20Holmes%7cSherlock%20Holmes%24%22%2c%20Options%3a%20Compiled).html>) 137.47 μs 123.80 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20Compiled).html>) 132.21 μs 119.66 μs 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes%22%2c%20Options%3a%20NonBacktracking).html>) 214.73 μs 202.97 μs 0.95 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20NonBacktracking).html>) 132.78 μs 119.21 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20None).html>) 153.78 μs 136.25 μs 0.89 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%22%2c%20Options%3a%20Compiled).html>) 139.18 μs 124.51 μs 0.89 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5c%5cs%2bHolmes%22%2c%20Options%3a%20None).html>) 148.68 μs 133.86 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fm)%5eSherlock%20Holmes%7cSherlock%20Holmes%24%22%2c%20Options%3a%20None).html>) 146.97 μs 130.49 μs 0.89 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%20Holmes%22%2c%20Options%3a%20Compiled).html>) 137.87 μs 124.36 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20Compiled).html>) 153.77 μs 135.19 μs 0.88 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22The%22%2c%20Options%3a%20Compiled).html>) 184.12 μs 168.58 μs 0.92 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes%22%2c%20Options%3a%20None).html>) 188.99 μs 172.40 μs 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22The%22%2c%20Options%3a%20None).html>) 216.66 μs 201.31 μs 0.93 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20None).html>) 132.77 μs 116.98 μs 0.88 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20None).html>) 132.34 μs 116.68 μs 0.88 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20Compiled).html>) 132.67 μs 118.87 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5c%5cs%2bHolmes%22%2c%20Options%3a%20NonBacktracking).html>) 164.56 μs 150.90 μs 0.92 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes%22%2c%20Options%3a%20Compiled).html>) 166.99 μs 149.17 μs 0.89 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%22%2c%20Options%3a%20NonBacktracking).html>) 150.99 μs 136.79 μs 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%22%2c%20Options%3a%20None).html>) 144.38 μs 130.78 μs 0.91 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20NonBacktracking).html>) 153.69 μs 137.09 μs 0.89 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5c%5cs%2bHolmes%22%2c%20Options%3a%20Compiled).html>) 140.42 μs 126.90 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20NonBacktracking).html>) 132.56 μs 118.73 μs 0.90 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bLu%7d%22%2c%20Options%3a%20None).html>) 2.64 ms 2.47 ms 0.93 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fs).*%22%2c%20Options%3a%20None).html>) 2.04 ms 1.84 ms 0.90 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fm)%5eSherlock%20Holmes%7cSherlock%20Holmes%24%22%2c%20Options%3a%20NonBacktracking).html>) 154.63 μs 140.23 μs 0.91 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%20Holmes%22%2c%20Options%3a%20NonBacktracking).html>) 154.37 μs 141.51 μs 0.92 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%20Holmes%22%2c%20Options%3a%20None).html>) 144.26 μs 130.85 μs 0.91 0.00 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 123.79793133074935 < 130.59049927150977. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 66.78599572562209 (T) = (0 -123528.3354847316) / Math.Sqrt((6331015.6141910935 / (299)) + (291278.75704263744 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09190887527364379 = (136030.77061451908 - 123528.3354847316) / 136030.77061451908 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 119.65614141791046 < 125.80125183364568. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 33.57705286845488 (T) = (0 -117577.15011783004) / Math.Sqrt((7061739.651517537 / (299)) + (2754919.046569531 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10101039660125388 = (130788.10886501297 - 117577.15011783004) / 130788.10886501297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 202.97027431781703 < 205.5861508176131. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 30.96332647249521 (T) = (0 -200618.1733276409) / Math.Sqrt((16812283.844781276 / (266)) + (2106524.493847712 / (21))) is greater than 1.968322603028609 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (266) + (21) - 2, .975) and 0.058744008858799664 = (213138.80093810274 - 200618.1733276409) / 213138.80093810274 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 119.20807010957792 < 126.08554078125. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 92.42005207275436 (T) = (0 -118935.90207003795) / Math.Sqrt((4672837.963734686 / (299)) + (74712.62308347275 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09717380499094194 = (131737.31857530415 - 118935.90207003795) / 131737.31857530415 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 136.25253173076925 < 146.11512262120326. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 50.964792124392616 (T) = (0 -135008.97634899267) / Math.Sqrt((7537790.314099853 / (299)) + (1921396.2073951047 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.11422842588542882 = (152419.6308556745 - 135008.97634899267) / 152419.6308556745 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 124.50539602864583 < 132.23477450958703. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 39.77528869513303 (T) = (0 -124393.15139604706) / Math.Sqrt((8772413.759751206 / (235)) + (1402336.687335298 / (21))) is greater than 1.9693475402190106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (235) + (21) - 2, .975) and 0.09352220783354533 = (137226.91550859858 - 124393.15139604706) / 137226.91550859858 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 133.8557769957983 < 141.04554603879416. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 43.8165166428013 (T) = (0 -133791.39530907915) / Math.Sqrt((9542079.750841701 / (262)) + (1021503.5872962303 / (20))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (262) + (20) - 2, .975) and 0.08831730656232734 = (146752.14992246183 - 133791.39530907915) / 146752.14992246183 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 130.48961249999996 < 139.55610849136988. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 50.23976955206207 (T) = (0 -132427.2386329546) / Math.Sqrt((9418567.510727257 / (296)) + (510299.24974310806 / (21))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (296) + (21) - 2, .975) and 0.0824615140799123 = (144328.81090558227 - 132427.2386329546) / 144328.81090558227 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 124.35500986842105 < 131.69616447826823. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 32.83327795123614 (T) = (0 -124022.8917960738) / Math.Sqrt((6171093.314794399 / (299)) + (2895577.9597101663 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0953538691137173 = (137095.4758570276 - 124022.8917960738) / 137095.4758570276 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 135.18563008040937 < 145.96807853680556. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 62.74472292645593 (T) = (0 -135858.6840470593) / Math.Sqrt((11001379.831209783 / (299)) + (575187.4104047571 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10474844025198915 = (151754.75827743864 - 135858.6840470593) / 151754.75827743864 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 168.57652327127664 < 174.71689346113445. IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 45.11507819352834 (T) = (0 -167460.56024845972) / Math.Sqrt((13087349.955442378 / (290)) + (1183537.8258212702 / (21))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (21) - 2, .975) and 0.07904164182138014 = (181832.93387949333 - 167460.56024845972) / 181832.93387949333 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 172.39641082875463 < 179.6801567005522. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 38.79093909412285 (T) = (0 -172817.24976358877) / Math.Sqrt((19805343.55060863 / (241)) + (1157343.5818858799 / (21))) is greater than 1.9691300033599934 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (241) + (21) - 2, .975) and 0.07678359792438394 = (187190.4023531789 - 172817.24976358877) / 187190.4023531789 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 201.31453199786324 < 205.83799980165526. IsChangePoint: Marked as a change because one of 10/26/2022 10:56:11 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 35.892583039635014 (T) = (0 -200814.40346210566) / Math.Sqrt((26009349.952135824 / (297)) + (933345.1120789277 / (21))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (297) + (21) - 2, .975) and 0.06098195082866415 = (213855.74392240943 - 200814.40346210566) / 213855.74392240943 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 116.98148328071183 < 126.01085880561439. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 30.270954082247822 (T) = (0 -117825.16030860276) / Math.Sqrt((8788457.447686095 / (299)) + (2898823.613747411 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.09687807926400673 = (130464.29015097089 - 117825.16030860276) / 130464.29015097089 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 116.68113483709274 < 125.91430949819608. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 29.32814738232518 (T) = (0 -118166.75429415652) / Math.Sqrt((7123424.815716272 / (299)) + (3385075.976274775 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09645934625582733 = (130781.88989558637 - 118166.75429415652) / 130781.88989558637 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 118.86830034722223 < 125.91635208783208. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 66.5340813202237 (T) = (0 -118494.76895762037) / Math.Sqrt((7211296.745413625 / (299)) + (237790.6455758246 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09560062991881291 = (131020.40191268953 - 118494.76895762037) / 131020.40191268953 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 150.89691774065233 < 156.42662999333933. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 60.47660290277976 (T) = (0 -149683.61554947362) / Math.Sqrt((8064129.288496213 / (299)) + (412282.1164269495 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08022348243954186 = (162739.11400400012 - 149683.61554947362) / 162739.11400400012 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 149.1701812271062 < 158.5178789755889. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 38.15341782649488 (T) = (0 -149846.59775649247) / Math.Sqrt((12842924.1890508 / (267)) + (1869923.3145079103 / (21))) is greater than 1.9682932552436228 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (267) + (21) - 2, .975) and 0.08616738256573979 = (163975.97863951515 - 149846.59775649247) / 163975.97863951515 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 136.79321586788728 < 143.53845678232997. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 58.199861454599215 (T) = (0 -136769.30070468082) / Math.Sqrt((9463078.29454491 / (262)) + (216731.63934868816 / (21))) is greater than 1.968442091833948 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (262) + (21) - 2, .975) and 0.08399853903911028 = (149311.22550962877 - 136769.30070468082) / 149311.22550962877 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 130.77818546831955 < 137.2530561620795. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 36.57875438104262 (T) = (0 -129472.7382988068) / Math.Sqrt((9920807.74308514 / (235)) + (1748095.9080354862 / (21))) is greater than 1.9693475402190106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (235) + (21) - 2, .975) and 0.09096643526338874 = (142428.99637740108 - 129472.7382988068) / 142428.99637740108 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 137.088002393018 < 146.0691066787027. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 8:11:05 PM, 1/17/2023 9:28:47 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 19.815338416840653 (T) = (0 -135868.62612084617) / Math.Sqrt((39835174.76272272 / (299)) + (1992215.3737731045 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11522013234755962 = (153562.06790887125 - 135868.62612084617) / 153562.06790887125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 126.89595021008405 < 133.3626870098586. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 37.299737642990735 (T) = (0 -124864.15365320224) / Math.Sqrt((8380079.095537045 / (285)) + (2302314.4775908403 / (21))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (21) - 2, .975) and 0.10022340411764964 = (138772.3955308666 - 124864.15365320224) / 138772.3955308666 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 118.72994479166668 < 125.91754639742233. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 76.99413679625758 (T) = (0 -118502.81237046972) / Math.Sqrt((7271824.398833005 / (299)) + (32877.564600126454 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0946417210505074 = (130890.5160816237 - 118502.81237046972) / 130890.5160816237 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Lu}", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.4655703755775575 < 2.5094661774468086. IsChangePoint: Marked as a change because one of 11/9/2022 1:54:51 PM, 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 73.9302401313585 (T) = (0 -2465786.7665568674) / Math.Sqrt((3467761809.371102 / (254)) + (2362346.0814658008 / (28))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (254) + (28) - 2, .975) and 0.10001132864210276 = (2739797.560825409 - 2465786.7665568674) / 2739797.560825409 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.840667666165413 < 1.9340128077248133. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 71.6803865596478 (T) = (0 -1842032.7354334383) / Math.Sqrt((5097171384.286157 / (299)) + (13041144.452585401 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1399878938328166 = (2141868.3786241417 - 1842032.7354334383) / 2141868.3786241417 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 140.2296702850877 < 146.32621401006364. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 43.053606187377255 (T) = (0 -139265.40598799413) / Math.Sqrt((11452179.631400721 / (299)) + (1082338.7250796878 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08480447580596409 = (152170.11262225936 - 139265.40598799413) / 152170.11262225936 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 141.5146674284141 < 146.42208557834667. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 45.81346274121131 (T) = (0 -139885.51506220057) / Math.Sqrt((6995971.044514563 / (299)) + (1107934.9912444802 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08288890354636869 = (152528.42932892495 - 139885.51506220057) / 152528.42932892495 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 130.84999667008202 < 136.9887540289837. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 44.385166891775455 (T) = (0 -130232.18290859132) / Math.Sqrt((8526324.99850907 / (296)) + (830816.4040408082 / (21))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (296) + (21) - 2, .975) and 0.08182238953473392 = (141837.6808846374 - 130232.18290859132) / 141837.6808846374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 14.05 μs 12.28 μs 0.87 0.10 False
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 14.22 μs 12.71 μs 0.89 0.17 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 12.284504536000002 < 13.247122205822619. IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 11/2/2022 9:17:08 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 41.065250616885706 (T) = (0 -12304.851324281875) / Math.Sqrt((232261.8524990056 / (299)) + (8696.121009400209 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10327482714262316 = (13721.9871781596 - 12304.851324281875) / 13721.9871781596 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 12.705051210000004 < 13.702732556650002. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 41.82296203292661 (T) = (0 -13011.934661376417) / Math.Sqrt((126361.50137851965 / (299)) + (7599.2533274251855 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08259011593542205 = (14183.338208355826 - 13011.934661376417) / 14183.338208355826 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20NonBacktracking).html>) 4.22 ms 3.83 ms 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Twain%22%2c%20Options%3a%20NonBacktracking).html>) 3.79 ms 3.48 ms 0.92 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20None).html>) 4.13 ms 3.76 ms 0.91 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5c%5cp%7bSm%7d%22%2c%20Options%3a%20None).html>) 37.66 ms 32.56 ms 0.86 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Twain%22%2c%20Options%3a%20Compiled).html>) 3.77 ms 3.42 ms 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20Compiled).html>) 4.04 ms 3.60 ms 0.89 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5c%5cp%7bSm%7d%22%2c%20Options%3a%20NonBacktracking).html>) 37.69 ms 32.59 ms 0.86 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Twain%22%2c%20Options%3a%20None).html>) 3.84 ms 3.37 ms 0.88 0.00 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.8339812802083335 < 4.007164275323772. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 31.40443579685322 (T) = (0 -3834551.3478965233) / Math.Sqrt((8672314259.673346 / (299)) + (1511393849.0676339 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07603928381207228 = (4150123.7885059607 - 3834551.3478965233) / 4150123.7885059607 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.478170238562091 < 3.6759650163072917. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 35.290358610016945 (T) = (0 -3514960.844332817) / Math.Sqrt((4885683736.915019 / (299)) + (1400622853.1831348 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08382053078082533 = (3836541.81568649 - 3514960.844332817) / 3836541.81568649 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.76119193125 < 3.9210299708834135. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 31.899490927142033 (T) = (0 -3746528.8066945635) / Math.Sqrt((8783916309.993465 / (299)) + (1403750726.3687615 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07708266868027423 = (4059441.3817510763 - 3746528.8066945635) / 4059441.3817510763 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 32.562206819047624 < 35.77587682857143. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/9/2022 4:06:23 PM, 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 76.41752962458898 (T) = (0 -32545461.097560402) / Math.Sqrt((4151564164871.7095 / (285)) + (273565723.5450341 / (29))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (29) - 2, .975) and 0.2208699143846809 = (41771536.86968406 - 32545461.097560402) / 41771536.86968406 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.42369797 < 3.5775999079575893. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 36.88693243003454 (T) = (0 -3389032.589611306) / Math.Sqrt((6112340622.098602 / (299)) + (1367720178.9868026 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09147451464042977 = (3730255.941328951 - 3389032.589611306) / 3730255.941328951 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.6002464173076922 < 3.821357521362036. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 29.77976344152622 (T) = (0 -3668217.350422953) / Math.Sqrt((8133424559.240875 / (299)) + (1545427112.577826 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07536240679078988 = (3967194.6905072196 - 3668217.350422953) / 3967194.6905072196 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 32.59097563809524 < 35.833277379107145. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/9/2022 4:06:23 PM, 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 67.87871835118808 (T) = (0 -32576277.912672546) / Math.Sqrt((5096090488268.23 / (299)) + (499990302.69441956 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.21393937988519546 = (41442449.95750425 - 32576277.912672546) / 41442449.95750425 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.368929324038461 < 3.6322373571986604. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 28.595514455357463 (T) = (0 -3448794.360004159) / Math.Sqrt((6067631426.342203 / (299)) + (2272946999.1916437 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0859237129270183 = (3772983.0745831397 - 3448794.360004159) / 3772983.0745831397 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayNot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Tests.Perf_BitArray.BitArrayNot(Size%3a%20512).html>) 23.53 ns 19.69 ns 0.84 0.33 False
[BitArrayXor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Tests.Perf_BitArray.BitArrayXor(Size%3a%20512).html>) 29.34 ns 24.91 ns 0.85 0.18 False
[BitArrayAnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size%3a%20512).html>) 29.81 ns 24.64 ns 0.83 0.19 False
[BitArrayOr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Tests.Perf_BitArray.BitArrayOr(Size%3a%20512).html>) 27.78 ns 25.09 ns 0.90 0.18 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.685207575976886 < 22.33822789201243. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 19.880785234460745 (T) = (0 -20.038725244131896) / Math.Sqrt((0.5506864650607074 / (299)) + (0.3809382626647422 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.12299351822723237 = (22.84900472300492 - 20.038725244131896) / 22.84900472300492 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.910302334612574 < 26.922638600413624. IsChangePoint: Marked as a change because one of 1/5/2023 10:03:03 PM, 1/13/2023 3:30:06 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 18.556275353104006 (T) = (0 -24.25888417214018) / Math.Sqrt((0.8367640687236114 / (299)) + (0.5585780133108246 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.118232598680264 = (27.511659124426753 - 24.25888417214018) / 27.511659124426753 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.637818395655597 < 27.063971268217152. IsChangePoint: Marked as a change because one of 12/8/2022 9:50:09 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 21.674402559170076 (T) = (0 -24.132442274989423) / Math.Sqrt((1.2257681073345026 / (299)) + (0.4629357451309509 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.12680696828687846 = (27.637007395310828 - 24.132442274989423) / 27.637007395310828 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.088824748149765 < 26.702354742540756. IsChangePoint: Marked as a change because one of 12/1/2022 2:43:16 PM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 22.582657509405696 (T) = (0 -24.153595746717148) / Math.Sqrt((0.8192547905924871 / (299)) + (0.4058645249533859 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1219502498986192 = (27.508231445801723 - 24.153595746717148) / 27.508231445801723 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options%3a%20Compiled).html>) 1.33 secs 1.21 secs 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options%3a%20NonBacktracking).html>) 1.40 secs 1.27 secs 0.91 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options%3a%20None).html>) 1.35 secs 1.22 secs 0.90 0.00 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.21174268975 < 1.2667465585. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 59.79369957033656 (T) = (0 -1210265848.4067512) / Math.Sqrt((679877179541627.9 / (299)) + (22685616206229.195 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08297444525394972 = (1319773306.3632095 - 1210265848.4067512) / 1319773306.3632095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.26887655925 < 1.3264597778166665. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 33.46320745439531 (T) = (0 -1267625250.6883) / Math.Sqrt((764792755715855.4 / (299)) + (180029084942632.44 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0809430130362089 = (1379267301.8852112 - 1267625250.6883) / 1379267301.8852112 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.219843125 < 1.2865644775854168. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 39.77316550646499 (T) = (0 -1227436232.5674815) / Math.Sqrt((595747188225323.8 / (299)) + (138010129616674.05 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08661497414079761 = (1343832226.0788736 - 1227436232.5674815) / 1343832226.0788736 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20Compiled).html>) 2.01 μs 1.76 μs 0.88 0.09 False
[MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20None).html>) 2.32 μs 2.10 μs 0.90 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.7635580600379481 < 1.899282255992434. IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 22.273287151926464 (T) = (0 -1753.697147263753) / Math.Sqrt((3505.0721321792876 / (299)) + (3487.427332858879 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1448237449170796 = (2050.6850334539627 - 1753.697147263753) / 2050.6850334539627 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.096849491695452 < 2.2197570461357743. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 27.307304235454357 (T) = (0 -2077.167372540673) / Math.Sqrt((2443.7544364610862 / (296)) + (1577.922639944638 / (21))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (296) + (21) - 2, .975) and 0.10718611732587255 = (2326.540181386078 - 2077.167372540673) / 2326.540181386078 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 63.04 ns 52.02 ns 0.83 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 52.01833022354032 < 59.739187533362454. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 60.681251539224036 (T) = (0 -52.615295561587274) / Math.Sqrt((0.9254569468932746 / (299)) + (0.4842674923252864 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.16007773057504482 = (62.643053383511116 - 52.615295561587274) / 62.643053383511116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare dad45a6f85dd2f254ff00cf29d5a48b1754b6a5d
Diff Diff

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Field_Get_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Reflection.Invoke.Field_Get_int.html>) 101.34 ns 90.67 ns 0.89 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Invoke.Field_Get_int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.67128785015561 < 96.3082050215486. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 17.9813407715121 (T) = (0 -88.9464481880318) / Math.Sqrt((4.483522599772606 / (299)) + (9.075286811205505 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.11908446544184953 = (100.97046163755705 - 88.9464481880318) / 100.97046163755705 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)