search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 6/7/2022 3:01:09 PM #6043

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5b%5e%5cn%5d*%22%2c%20Options%3a%20NonBacktracking).html>) 17.48 ms 8.99 ms 0.51 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cb%5cw%2bn%5cb%22%2c%20Options%3a%20NonBacktracking).html>) 12.17 ms 8.69 ms 0.71 0.19 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-zA-Z%5d%2bing%22%2c%20Options%3a%20NonBacktracking).html>) 9.78 ms 6.23 ms 0.64 0.07 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cw%2b%5cs%2bHolmes%5cs%2b%5cw%2b%22%2c%20Options%3a%20NonBacktracking).html>) 5.11 ms 3.44 ms 0.67 0.05 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20NonBacktracking).html>) 253.81 μs 210.24 μs 0.83 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-q%5d%5b%5eu-z%5d%7b13%7dx%22%2c%20Options%3a%20NonBacktracking).html>) 108.50 μs 88.65 μs 0.82 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cw%2b%22%2c%20Options%3a%20NonBacktracking).html>) 25.88 ms 19.17 ms 0.74 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22.*%22%2c%20Options%3a%20NonBacktracking).html>) 17.10 ms 8.83 ms 0.52 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fs).*%22%2c%20Options%3a%20NonBacktracking).html>) 14.37 ms 6.37 ms 0.44 0.08 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%5cs%2b%5cw%2b%22%2c%20Options%3a%20NonBacktracking).html>) 2.18 ms 1.57 ms 0.72 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes.%7b0%2c25%7dWatson%7cWatson.%7b0%2c25%7dHolmes%22%2c%20Options%3a%20NonBacktracking).html>) 235.29 μs 179.17 μs 0.76 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5cw%2b%5cs%2bHolmes%22%2c%20Options%3a%20NonBacktracking).html>) 5.14 ms 3.47 ms 0.68 0.01 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.993287380952381 < 16.146394854395602. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 24.46424919963778 (T) = (0 -8926594.514391292) / Math.Sqrt((6744215256938.665 / (299)) + (6277211701.665101 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.2922669796329656 = (12612940.554563794 - 8926594.514391292) / 12612940.554563794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.693283497536946 < 11.568112135714285. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 52.50705466514946 (T) = (0 -8646730.332364095) / Math.Sqrt((519662669713.1273 / (299)) + (158056965253.23297 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.31053432353572485 = (12541204.917852249 - 8646730.332364095) / 12541204.917852249 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.225887666666667 < 8.734669396113148. IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 241.11625005556033 (T) = (0 -6252380.745874682) / Math.Sqrt((26451906807.368942 / (299)) + (2744381211.291692 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.32244837653905856 = (9227903.128528347 - 6252380.745874682) / 9227903.128528347 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.443669757639621 < 4.860795661078717. IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 180.87623311612745 (T) = (0 -3444939.4292770163) / Math.Sqrt((24101338071.943676 / (299)) + (195888074.25763136 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.3265271866817718 = (5115187.073853298 - 3444939.4292770163) / 5115187.073853298 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 210.24238295486404 < 242.1705097069597. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 47.122473851487555 (T) = (0 -209299.82931791552) / Math.Sqrt((84376756.49254225 / (299)) + (4325065.595160721 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.12294876823899363 = (238640.3687019153 - 209299.82931791552) / 238640.3687019153 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 88.65426178511605 < 104.02276503210742. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 55.74226258590348 (T) = (0 -87813.47461207573) / Math.Sqrt((38713994.852505505 / (299)) + (814987.2067645632 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.19674328388048654 = (109321.80565672395 - 87813.47461207573) / 109321.80565672395 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.168016666666666 < 24.50130591666667. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 31.0116453805795 (T) = (0 -19341008.806689337) / Math.Sqrt((3002352375596.946 / (299)) + (18128476586.439213 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.14095931562894856 = (22514659.85088925 - 19341008.806689337) / 22514659.85088925 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.825741904761905 < 16.04948174404762. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 22.917750959943632 (T) = (0 -8917907.830683103) / Math.Sqrt((7269643252950.65 / (299)) + (28082737305.185028 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.2889217316659349 = (12541387.112808604 - 8917907.830683103) / 12541387.112808604 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.37163641025641 < 13.614854315476187. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 27.66829352469571 (T) = (0 -6373877.386527785) / Math.Sqrt((5791437739209.848 / (299)) + (13374484300.68226 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.3785284270514757 = (10256104.484855859 - 6373877.386527785) / 10256104.484855859 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.5677005390835579 < 2.072937063768116. IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 65.14004558934101 (T) = (0 -1585563.562391735) / Math.Sqrt((10442024135.698298 / (299)) + (121547636.81305227 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.20183867934604213 = (1986520.169998509 - 1585563.562391735) / 1986520.169998509 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 179.16726924916307 < 219.0964342787435. IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 87.62598879471842 (T) = (0 -179645.5687836314) / Math.Sqrt((19066456.127906267 / (299)) + (11454715.198137904 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.22223293490784177 = (230976.05548821363 - 179645.5687836314) / 230976.05548821363 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.4734353703703706 < 4.883556667648607. IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 154.18065622059675 (T) = (0 -3478962.40146624) / Math.Sqrt((29577616884.481995 / (299)) + (954454113.636407 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.32783794272850747 = (5175779.209538236 - 3478962.40146624) / 5175779.209538236 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%209%2c%20Options%3a%20NonBacktracking).html>) 176.75 ns 134.79 ns 0.76 0.01 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%205%2c%20Options%3a%20NonBacktracking).html>) 286.48 ns 231.83 ns 0.81 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%204%2c%20Options%3a%20NonBacktracking).html>) 276.46 ns 201.48 ns 0.73 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%202%2c%20Options%3a%20NonBacktracking).html>) 269.61 ns 220.61 ns 0.82 0.01 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%203%2c%20Options%3a%20NonBacktracking).html>) 350.02 ns 263.85 ns 0.75 0.04 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%206%2c%20Options%3a%20NonBacktracking).html>) 181.58 ns 137.70 ns 0.76 0.02 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%208%2c%20Options%3a%20NonBacktracking).html>) 164.41 ns 134.39 ns 0.82 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2010%2c%20Options%3a%20NonBacktracking).html>) 195.32 ns 142.65 ns 0.73 0.01 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 134.79119070163398 < 171.47615808203412. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 94.53526319671155 (T) = (0 -134.20121802151732) / Math.Sqrt((36.81305072250611 / (299)) + (2.2271060064757284 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.22818540973261706 = (173.87753446721632 - 134.20121802151732) / 173.87753446721632 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 231.82910058170063 < 276.70117454926014. IsChangePoint: Marked as a change because one of 4/19/2022 10:22:09 AM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 17.745546084312316 (T) = (0 -232.36270728711878) / Math.Sqrt((378.41162695134125 / (299)) + (8.887101179050278 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.08493609778667814 = (253.9305798481272 - 232.36270728711878) / 253.9305798481272 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 201.47979392523987 < 257.4851053391598. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 19.9614961746299 (T) = (0 -208.11301775754131) / Math.Sqrt((384.0244039295436 / (299)) + (15.254396254305888 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.1096200868026163 = (233.73507721013223 - 208.11301775754131) / 233.73507721013223 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 220.61331926510346 < 256.1188621492222. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 49.86918583814333 (T) = (0 -222.2179382475268) / Math.Sqrt((58.22036881729944 / (299)) + (11.248247685486298 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.13322236099420576 = (256.37248614582836 - 222.2179382475268) / 256.37248614582836 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 263.8538038753821 < 332.36167929819624. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 36.81234050575533 (T) = (0 -268.023117105912) / Math.Sqrt((387.1881828480365 / (299)) + (11.684466957283423 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.14722300539407926 = (314.29449762510177 - 268.023117105912) / 314.29449762510177 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 137.69612672922096 < 172.45453162416146. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 71.72699602635717 (T) = (0 -136.95235902143835) / Math.Sqrt((39.34898096962986 / (299)) + (5.161663998703834 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.20806273224388247 = (172.93334282585343 - 136.95235902143835) / 172.93334282585343 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 134.38965118361835 < 152.84429425230365. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/23/2022 3:43:39 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 27.45364639595364 (T) = (0 -133.3449329365982) / Math.Sqrt((52.884037923009124 / (299)) + (9.440022713211713 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.11541854072344525 = (150.74353134832026 - 133.3449329365982) / 150.74353134832026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 142.65043557293274 < 188.97476760451127. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 76.53355730027859 (T) = (0 -144.2934204464586) / Math.Sqrt((75.91460187204896 / (299)) + (3.2360063342897014 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.23330562323810058 = (188.20200697946376 - 144.2934204464586) / 188.20200697946376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b0%2c2%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20NonBacktracking).html>) 128.29 ms 85.53 ms 0.67 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b2%2c4%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20NonBacktracking).html>) 128.13 ms 85.81 ms 0.67 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 85.53031923076924 < 122.41787875. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 114.46929630619033 (T) = (0 -85125476.4840489) / Math.Sqrt((35571660666898.91 / (299)) + (1495540174783.641 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.3452380807452554 = (130009815.75247905 - 85125476.4840489) / 130009815.75247905 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 85.8072 < 121.7366845297619. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 149.9228334665284 (T) = (0 -85328809.89839524) / Math.Sqrt((12784292173617.58 / (299)) + (1563721109429.5327 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.3319610198298689 = (127730285.85347572 - 85328809.89839524) / 127730285.85347572 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CheckArrayIsInterfaceNo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo.html>) 482.69 μs 371.20 μs 0.77 0.43 False
[IFooObjIsIFoo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.CastingPerf.IFooObjIsIFoo.html>) 751.03 μs 688.25 μs 0.92 0.39 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 371.20021802325584 < 588.529661751443. IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/16/2022 12:27:40 PM, 6/4/2022 3:35:14 AM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 4.492542633038725 (T) = (0 -426045.2183276256) / Math.Sqrt((38175244157.05323 / (299)) + (9965044348.481388 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.16876649447737874 = (512545.77143128787 - 426045.2183276256) / 512545.77143128787 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.CastingPerf.IFooObjIsIFoo ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 688.2523809523809 < 711.7827376302083. IsChangePoint: Marked as a change because one of 5/14/2022 2:55:25 AM, 5/15/2022 9:56:58 AM, 6/7/2022 9:37:06 AM, 6/8/2022 12:13:12 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 3.53432498228283 (T) = (0 -567435.0734554838) / Math.Sqrt((25888727093.20902 / (299)) + (14602988490.352875 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.1190655307126767 = (644128.5853129793 - 567435.0734554838) / 644128.5853129793 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Int32).IndexOfValue(Size%3a%20512).html>) 28.45 ns 25.79 ns 0.91 0.18 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.IndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.7928939842001 < 26.96585772802251. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/11/2022 9:09:01 PM, 5/23/2022 3:43:39 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 11.409984573949874 (T) = (0 -25.61722639681535) / Math.Sqrt((1.8678974890519977 / (299)) + (0.5356797315796962 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.05828942075843352 = (27.20286568028886 - 25.61722639681535) / 27.20286568028886 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22%5b%5cw%5d%2b%3a%2f%2f%5b%5e%2f%5cs%3f%23%5d%2b%5b%5e%5cs%3f%23%5d%2b(%3f%3a%5c%3f%5b%5e%5cs%23%5d)%3f(%3f%3a%23%5b%5e%5cs%5d)%3f%22%2c%20Options%3a%20NonBacktracking).html>) 8.85 ms 3.93 ms 0.44 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22(%3f%3a(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%5c.)%7b3%7d(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%22%2c%20Options%3a%20NonBacktracking).html>) 18.93 ms 17.77 ms 0.94 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.926008049886621 < 8.537157093253967. IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 21.407332443141776 (T) = (0 -3867831.1080268556) / Math.Sqrt((2649271317491.1562 / (299)) + (6297518594.594724 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.3444220490452514 = (5899879.79674111 - 3867831.1080268556) / 5899879.79674111 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.76816666666667 < 18.071337. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 5:23:46 PM, 5/19/2022 10:26:24 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsImprovementStdDev: Marked as improvement because 25.059653940544226 (T) = (0 -17898128.982025728) / Math.Sqrt((608702858265.0735 / (299)) + (133549101401.93297 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.09123618603325842 = (19695028.242707685 - 17898128.982025728) / 19695028.242707685 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)