search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/16/2022 10:12:41 PM #5209

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22foobardzsdzs%22%2c%20value%3a%20%22rddzs%22%2c%20comparisonType%3a%20InvariantCulture).html>) 26.63 ns 23.08 ns 0.87 0.06 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22string1%22%2c%20value%3a%20%22string2%22%2c%20comparisonType%3a%20InvariantCulture).html>) 21.41 ns 18.06 ns 0.84 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 23.081733315573157 < 25.31866095837981. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 94.38556283891967 (T) = (0 -23.098074352421783) / Math.Sqrt((0.4582510503587043 / (299)) + (0.0001944914068107007 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1385940510170486 = (26.814389173528834 - 23.098074352421783) / 26.814389173528834 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.06256171105653 < 20.345896481559265. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 70.991402027774 (T) = (0 -18.06320264254752) / Math.Sqrt((0.8096562038892611 / (299)) + (0.0001391413879398544 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17011923378286276 = (21.766021551367423 - 18.06320264254752) / 21.766021551367423 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5cs%2bHolmes%22%2c%20Options%3a%20Compiled).html>) 47.00 μs 40.35 μs 0.86 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20NonBacktracking).html>) 44.84 μs 36.02 μs 0.80 0.25 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%20Holmes%22%2c%20Options%3a%20Compiled).html>) 47.27 μs 40.04 μs 0.85 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20Compiled).html>) 42.14 μs 34.38 μs 0.82 0.46 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5cs%2bHolmes%22%2c%20Options%3a%20None).html>) 51.23 μs 44.44 μs 0.87 0.08 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20Compiled).html>) 58.15 μs 48.91 μs 0.84 0.11 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fm)%5eSherlock%20Holmes%7cSherlock%20Holmes%24%22%2c%20Options%3a%20Compiled).html>) 46.41 μs 38.98 μs 0.84 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20None).html>) 58.05 μs 49.26 μs 0.85 0.06 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%20Holmes%22%2c%20Options%3a%20NonBacktracking).html>) 70.86 μs 63.90 μs 0.90 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fm)%5eSherlock%20Holmes%7cSherlock%20Holmes%24%22%2c%20Options%3a%20NonBacktracking).html>) 66.29 μs 59.10 μs 0.89 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5cs%2bHolmes%22%2c%20Options%3a%20NonBacktracking).html>) 84.89 μs 80.25 μs 0.95 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20None).html>) 45.12 μs 34.72 μs 0.77 0.43 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20None).html>) 43.16 μs 37.13 μs 0.86 0.41 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20Compiled).html>) 44.67 μs 36.09 μs 0.81 0.39 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20NonBacktracking).html>) 44.94 μs 34.89 μs 0.78 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 40.34504154914531 < 44.596727351621745. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 84.91022473647656 (T) = (0 -40354.61361829835) / Math.Sqrt((1236143.6800785495 / (299)) + (42878.97249115518 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15865758649319317 = (47964.553991871064 - 40354.61361829835) / 47964.553991871064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 36.01979996685449 < 42.67522238401024. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 3/11/2022 8:42:43 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 50.11034821666803 (T) = (0 -36139.17514655944) / Math.Sqrt((3920335.7073505013 / (299)) + (31926.7333376392 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1492750411709372 = (42480.445379551784 - 36139.17514655944) / 42480.445379551784 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 40.04037733843538 < 44.226989162461045. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 97.87196663756089 (T) = (0 -40031.92063871508) / Math.Sqrt((534968.2385443865 / (299)) + (30774.331074763923 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1420584949538895 = (46660.43128029287 - 40031.92063871508) / 46660.43128029287 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.3770678872981 < 42.501602008556546. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 56.43302819130479 (T) = (0 -34865.39786285512) / Math.Sqrt((2168096.15948833 / (299)) + (186092.21116896323 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20104270753547163 = (43638.6252327787 - 34865.39786285512) / 43638.6252327787 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 44.43885228531855 < 50.75090770377448. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 73.75273249387541 (T) = (0 -43755.36995873893) / Math.Sqrt((1453349.1297227866 / (299)) + (125552.1070611712 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17697568877343048 = (53164.12815744097 - 43755.36995873893) / 53164.12815744097 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 48.91080880859376 < 52.79997957866325. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 41.64591170757317 (T) = (0 -49161.95823346055) / Math.Sqrt((2054525.9223224684 / (299)) + (317396.63015452604 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13801668786008697 = (57033.53828441729 - 49161.95823346055) / 57033.53828441729 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 38.982422673092856 < 44.090957657246896. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 65.48415659378253 (T) = (0 -39159.24300111845) / Math.Sqrt((458314.9423997015 / (299)) + (111395.47858970519 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1529514835349131 = (46230.22440855959 - 39159.24300111845) / 46230.22440855959 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 49.25982151280325 < 55.17494518842753. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 42.2166150151016 (T) = (0 -49339.756433463255) / Math.Sqrt((1809674.8872484856 / (299)) + (306572.85268008604 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13613705565409764 = (57115.259725398006 - 49339.756433463255) / 57115.259725398006 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 63.89968768147503 < 67.4676065681561. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/12/2022 1:03:18 PM, 2/18/2022 5:51:51 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 23.460402472680734 (T) = (0 -64168.420483282884) / Math.Sqrt((645533.5278758339 / (299)) + (855668.1919120711 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09368974727486355 = (70801.82563348285 - 64168.420483282884) / 70801.82563348285 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 59.10064748427672 < 62.74469156304848. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/12/2022 1:03:18 PM, 2/18/2022 5:51:51 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 27.972901560568648 (T) = (0 -59908.629193304485) / Math.Sqrt((869768.8685046935 / (299)) + (646754.6003071334 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10393184739100224 = (66857.2239944854 - 59908.629193304485) / 66857.2239944854 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 80.25378079427084 < 82.05681921038739. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 45.58066224374304 (T) = (0 -78890.47769407924) / Math.Sqrt((1479203.6424538363 / (299)) + (250015.59810027078 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08768981092110464 = (86473.30550339485 - 78890.47769407924) / 86473.30550339485 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.723448004434594 < 40.683307480476344. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 50.62649843534924 (T) = (0 -35009.872893156244) / Math.Sqrt((2235827.29678215 / (299)) + (247234.69275846626 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.200173801561916 = (43771.850636456016 - 35009.872893156244) / 43771.850636456016 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 37.131174037356324 < 41.811150594601514. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 47.43882496995491 (T) = (0 -36284.18383982647) / Math.Sqrt((2021091.1812252046 / (299)) + (191463.98566160898 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.168911786552963 = (43658.64327365866 - 36284.18383982647) / 43658.64327365866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 36.093689923371656 < 42.45931431220238. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 83.22339903804983 (T) = (0 -36028.667431823334) / Math.Sqrt((1972759.9795058798 / (299)) + (24828.762462482475 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17855449000355955 = (43860.08200590135 - 36028.667431823334) / 43860.08200590135 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.889348251488094 < 40.32441367702737. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 3/11/2022 8:42:43 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 39.07575659758761 (T) = (0 -34896.75727018755) / Math.Sqrt((3825595.1664961535 / (299)) + (248642.65908031235 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1740150971183182 = (42248.66235259306 - 34896.75727018755) / 42248.66235259306 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetMethod20 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod20.html>) 1.59 ms 1.36 ms 0.86 0.07 False
[GetMethod10 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod10.html>) 790.13 μs 650.02 μs 0.82 0.05 False
[GetMethod3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod3.html>) 239.70 μs 197.41 μs 0.82 0.05 False
[GetMethod2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod2.html>) 162.14 μs 126.13 μs 0.78 0.08 False
[GetMethod4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod4.html>) 317.22 μs 260.78 μs 0.82 0.04 False
[GetMethod1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod1.html>) 81.55 μs 63.84 μs 0.78 0.12 False
[GetMethod12 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/PerfLabTests.GetMember.GetMethod12.html>) 964.17 μs 773.78 μs 0.80 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.GetMember.GetMethod20 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.3627835991319444 < 1.5137071908958335. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 27.104055904904467 (T) = (0 -1330158.1145069555) / Math.Sqrt((900155134.5242589 / (299)) + (1056599638.5918795 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.16861411498282766 = (1599928.6714850601 - 1330158.1145069555) / 1599928.6714850601 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod10 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 650.0162180989583 < 751.0139764169814. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 30.597236745414953 (T) = (0 -656368.3726157252) / Math.Sqrt((345581886.93496615 / (299)) + (209322033.01834637 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17316777059849014 = (793835.0118388921 - 656368.3726157252) / 793835.0118388921 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 197.4136770071138 < 230.19241316114923. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 42.03399676529376 (T) = (0 -194205.08102637876) / Math.Sqrt((27284798.59429786 / (299)) + (10235066.82559034 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1795064196960367 = (236692.992715961 - 194205.08102637876) / 236692.992715961 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 126.13343163798922 < 150.78700382634392. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 67.00258922540242 (T) = (0 -127049.67067332401) / Math.Sqrt((13940751.954483673 / (299)) + (1763672.3482522543 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19349389630499622 = (157530.94749220938 - 127049.67067332401) / 157530.94749220938 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 260.7792226092896 < 302.87373652321605. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 39.8530557295613 (T) = (0 -258642.288104424) / Math.Sqrt((39218696.77684502 / (299)) + (21950958.439354483 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18347796785509252 = (316760.9420470885 - 258642.288104424) / 316760.9420470885 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod1 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 63.836219071161636 < 75.98259215052083. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 3/18/2022 1:59:24 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 56.22130218548419 (T) = (0 -63849.51974393269) / Math.Sqrt((4611782.151777028 / (299)) + (519546.61727544654 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18060040458914203 = (77922.32276111594 - 63849.51974393269) / 77922.32276111594 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod12 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 773.7811531593406 < 930.8044830110676. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 42.70221405676925 (T) = (0 -786840.2316644615) / Math.Sqrt((331573656.9417316 / (299)) + (157077247.923438 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1755260650865575 = (954354.2837980294 - 786840.2316644615) / 954354.2837980294 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Trim_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Trim_CharArr(s%3a%20%22%20Te%20st%20%20%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 15.41 ns 13.16 ns 0.85 0.08 False
[Trim_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Trim_CharArr(s%3a%20%22%20Test%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 13.47 ns 11.66 ns 0.87 0.17 False
[Trim_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Tests.Perf_String.Trim_CharArr(s%3a%20%22Test%20%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 13.64 ns 11.59 ns 0.85 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' ']) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.158742142221115 < 14.640948171694763. IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/3/2022 4:53:52 PM, 3/3/2022 7:35:51 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 49.15032640022461 (T) = (0 -13.050233823818536) / Math.Sqrt((0.19369079335204176 / (299)) + (0.011388277649608952 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13383316667163173 = (15.066651506005108 - 13.050233823818536) / 15.066651506005108 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' ']) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.659337118147656 < 12.79818581563892. IsChangePoint: Marked as a change because one of 2/1/2022 10:36:16 AM, 2/3/2022 4:53:52 PM, 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 26.459719440783985 (T) = (0 -11.540255734520796) / Math.Sqrt((0.4745718005878263 / (299)) + (0.020914859478469976 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11927089350154434 = (13.103070682427857 - 11.540255734520796) / 13.103070682427857 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' ']) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.59067932672317 < 12.913663771205051. IsChangePoint: Marked as a change because one of 2/1/2022 1:24:52 PM, 2/3/2022 4:53:52 PM, 3/3/2022 7:35:51 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 30.675297422068706 (T) = (0 -11.539947823383294) / Math.Sqrt((0.43315315500333507 / (299)) + (0.00831214431115605 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11095485111486306 = (12.980159486674433 - 11.539947823383294) / 12.980159486674433 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.CtorFromCollection(Int32).ImmutableSortedSet(Size%3a%20512).html>) 16.51 μs 11.97 μs 0.72 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.96597855326723 < 15.161683950671852. IsChangePoint: Marked as a change because one of 3/4/2022 7:00:20 AM, 3/14/2022 4:00:26 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 18.103138176960183 (T) = (0 -12205.617442417033) / Math.Sqrt((851003.2983343917 / (299)) + (222664.02915079 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18391804087303773 = (14956.362294143228 - 12205.617442417033) / 14956.362294143228 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.IO.Tests.Perf_StreamWriter.WriteFormat.html>) 66.14 ns 57.79 ns 0.87 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_StreamWriter.WriteFormat ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 57.79403274530833 < 62.97345016925957. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 26.027768438261095 (T) = (0 -58.05673862281605) / Math.Sqrt((1.72253014057786 / (299)) + (0.8129403331184729 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11232377201206603 = (65.40305664646593 - 58.05673862281605) / 65.40305664646593 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HtmlDecode_Entities - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities.html>) 121.67 ns 112.98 ns 0.93 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 112.98017915446836 < 115.84604825313386. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 15.949577821942157 (T) = (0 -114.07056093795724) / Math.Sqrt((5.507418878176152 / (299)) + (3.5084869758559463 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07511367652859759 = (123.33468237460029 - 114.07056093795724) / 123.33468237460029 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options%3a%20None).html>) 505.90 ms 426.64 ms 0.84 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 426.6358626666667 < 472.34583775. IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 83.49800882953842 (T) = (0 -425298767.4431818) / Math.Sqrt((106896796266248.05 / (299)) + (5147119922497.89 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1513705432080119 = (501159562.67993295 - 425298767.4431818) / 501159562.67993295 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsKeyFalse(String%2c%20String).IDictionary(Size%3a%20512).html>) 9.73 μs 8.08 μs 0.83 0.02 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.ContainsKeyFalse(String%2c%20String).Dictionary(Size%3a%20512).html>) 8.65 μs 7.64 μs 0.88 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<String, String>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.08171458119219 < 9.239119238790275. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 85.4354549734524 (T) = (0 -8115.802033008475) / Math.Sqrt((62261.376535330004 / (299)) + (2266.3527349053966 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1764545665646082 = (9854.71074638066 - 8115.802033008475) / 9854.71074638066 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyFalse<String, String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.637488730831316 < 8.367546664133682. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 53.97814543832588 (T) = (0 -7454.8002704556675) / Math.Sqrt((49986.828362259854 / (299)) + (5155.018369074243 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15438979563394326 = (8815.882580372167 - 7454.8002704556675) / 8815.882580372167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%20-2147483648).html>) 11.67 ns 10.34 ns 0.89 0.02 True
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%2012345).html>) 9.72 ns 8.35 ns 0.86 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.335622240681325 < 11.06999752959778. IsChangePoint: Marked as a change because one of 4/14/2022 12:09:31 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 87.13086780005841 (T) = (0 -10.332259914131184) / Math.Sqrt((0.05967289332590734 / (299)) + (0.0005567213678985864 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11768755990086578 = (11.710432092479937 - 10.332259914131184) / 11.710432092479937 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.35243640506557 < 9.229780235520328. IsChangePoint: Marked as a change because one of 2/1/2022 5:51:01 AM, 2/8/2022 5:20:12 PM, 2/23/2022 5:12:02 AM, 4/14/2022 12:09:31 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 12.691217002089276 (T) = (0 -8.626536275589938) / Math.Sqrt((0.10846622936871321 / (299)) + (0.11563644285883637 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09932593209467876 = (9.577866825512688 - 8.626536275589938) / 9.577866825512688 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateFor(String).ReadOnlySpan(Size%3a%20512).html>) 228.53 ns 174.26 ns 0.76 0.04 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IterateFor(String).Span(Size%3a%20512).html>) 231.95 ns 179.96 ns 0.78 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 174.2648671923244 < 217.2037759676721. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/19/2022 11:51:48 PM, 2/23/2022 6:59:06 PM, 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/14/2022 4:00:26 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 18.783419200217242 (T) = (0 -174.65886532019638) / Math.Sqrt((730.9613186306797 / (299)) + (0.5323591922554222 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1451572168238938 = (204.31694430555297 - 174.65886532019638) / 204.31694430555297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 179.95802615024635 < 220.25168057802642. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/19/2022 11:51:48 PM, 2/23/2022 6:59:06 PM, 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/14/2022 4:00:26 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 18.880741488297698 (T) = (0 -180.44462015465254) / Math.Sqrt((660.7633255531516 / (299)) + (0.5953398271528849 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13602469855436053 = (208.85391035221156 - 180.44462015465254) / 208.85391035221156 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 21.00 ns 17.43 ns 0.83 0.15 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 53.05 ns 46.45 ns 0.88 0.11 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 52.61 ns 46.74 ns 0.89 0.11 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 102.44 ns 88.78 ns 0.87 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.43136411720841 < 19.41020560561606. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 23.488776504894478 (T) = (0 -16.62741375164637) / Math.Sqrt((1.9128331471794637 / (299)) + (0.17515411756849233 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1742711405798206 = (20.1366508654209 - 16.62741375164637) / 20.1366508654209 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 46.450100008808974 < 50.39528630827502. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/14/2022 10:40:48 AM, 4/14/2022 8:50:18 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 34.626325785812114 (T) = (0 -46.44492029617603) / Math.Sqrt((6.292296878599911 / (299)) + (0.0009040347367833193 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.09769178038590152 = (51.47345362324164 - 46.44492029617603) / 51.47345362324164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 46.7368991423391 < 49.95191785231071. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 4/14/2022 8:50:18 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 31.9123675206555 (T) = (0 -46.77976029499646) / Math.Sqrt((6.092007462661852 / (299)) + (0.002512895269565432 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.08898301244065114 = (51.34894401949772 - 46.77976029499646) / 51.34894401949772 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 88.78312735541243 < 96.68531905228082. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/17/2022 3:08:05 PM, 4/14/2022 8:50:18 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 29.11694377703815 (T) = (0 -89.6542562322558) / Math.Sqrt((26.396080772431603 / (299)) + (0.3322869286787765 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.09517403241268285 = (99.08453055488157 - 89.6542562322558) / 99.08453055488157 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.Sort<IntClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.Sort(IntClass).Array_Comparison(Size%3a%20512).html>) 38.93 μs 35.07 μs 0.90 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntClass&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntClass>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.06761388571428 < 37.042141638000004. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/22/2022 1:01:28 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 24.123729130596473 (T) = (0 -35324.12737696969) / Math.Sqrt((2202787.5315271006 / (299)) + (36604.76756511147 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06596652028526367 = (37818.90921914068 - 35324.12737696969) / 37818.90921914068 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Twain%22%2c%20Options%3a%20NonBacktracking).html>) 1.71 ms 1.56 ms 0.91 0.44 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20Compiled).html>) 6.22 ms 1.62 ms 0.26 0.02 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20NonBacktracking).html>) 8.34 ms 1.81 ms 0.22 0.02 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20None).html>) 8.17 ms 1.80 ms 0.22 0.03 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.5608952533333336 < 1.75137092755012. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 18.512545052526914 (T) = (0 -1568896.5519958066) / Math.Sqrt((4893939764.035051 / (299)) + (1160785803.7427652 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11525987093780592 = (1773285.1720638059 - 1568896.5519958066) / 1773285.1720638059 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.6219919445833333 < 5.90129857595486. IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 219.1433796516236 (T) = (0 -1719134.1181548787) / Math.Sqrt((3502008401.53809 / (299)) + (10780864196.965605 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.7246794624848859 = (6244118.704949515 - 1719134.1181548787) / 6244118.704949515 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.8108077768518516 < 7.878925951071428. IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 268.14045405225085 (T) = (0 -1915985.8036055004) / Math.Sqrt((10523435176.812702 / (299)) + (14281624554.464806 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.771868519020109 = (8398603.276390372 - 1915985.8036055004) / 8398603.276390372 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.7954905691964287 < 7.766449894006409. IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 258.2320921478184 (T) = (0 -1793864.7068952122) / Math.Sqrt((14919720626.004145 / (299)) + (15295227554.726206 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.7843229565126995 = (8317365.065331291 - 1793864.7068952122) / 8317365.065331291 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.CtorFromCollection(String).HashSet(Size%3a%20512).html>) 10.66 μs 9.93 μs 0.93 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.925158616609293 < 10.1305372484816. IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/24/2022 12:54:34 PM, 3/17/2022 6:01:08 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 29.971238913959326 (T) = (0 -9939.170717788162) / Math.Sqrt((179297.102638361 / (299)) + (3769.8471859389474 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08472591412179782 = (10859.228804944874 - 9939.170717788162) / 10859.228804944874 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.Sort(Int32).Array(Size%3a%20512).html>) 4.37 μs 2.46 μs 0.56 0.62 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.Sort(Int32).List(Size%3a%20512).html>) 6.14 μs 2.71 μs 0.44 0.61 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.4626081400000013 < 4.19826829575. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 29.386372245089305 (T) = (0 -2489.7158725518343) / Math.Sqrt((720343.2771690802 / (299)) + (14673.103207242752 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.41932157298184314 = (4287.598362034526 - 2489.7158725518343) / 4287.598362034526 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.712692071428571 < 3.6121846817500005. IsChangePoint: Marked as a change because one of 4/12/2022 5:02:33 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 25.431750501951917 (T) = (0 -2625.623360088858) / Math.Sqrt((662513.6554654047 / (299)) + (19465.856558164895 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37945073573212823 = (4231.1279881809005 - 2625.623360088858) / 4231.1279881809005 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 1.36 ms 1.28 ms 0.94 0.00 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 1.47 ms 1.34 ms 0.91 0.00 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 1.36 ms 1.27 ms 0.93 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.278818210576923 < 1.2969077075750246. IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 4/14/2022 12:09:31 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 70.61739566197004 (T) = (0 -1279627.3871649185) / Math.Sqrt((266311801.6666935 / (299)) + (6618408.593370981 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0631581731196738 = (1365894.807905903 - 1279627.3871649185) / 1365894.807905903 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.3399437972222223 < 1.4332221968580896. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 75.77654565901439 (T) = (0 -1338776.3164311557) / Math.Sqrt((999952925.010694 / (299)) + (2614895.2465815055 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09675988264043907 = (1482193.1518551195 - 1338776.3164311557) / 1482193.1518551195 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.2707804983974358 < 1.287743394233631. IsChangePoint: Marked as a change because one of 3/8/2022 5:44:17 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 100.0674738898958 (T) = (0 -1270282.3844080598) / Math.Sqrt((231103633.10396826 / (299)) + (2049171.9309868317 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07162627981510702 = (1368287.745322081 - 1270282.3844080598) / 1368287.745322081 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.12/System.Collections.IndexerSet(String).Span(Size%3a%20512).html>) 232.60 ns 180.43 ns 0.78 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 180.43475644522294 < 220.87151498831088. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM. IsImprovementStdDev: Marked as improvement because 6.769667285394283 (T) = (0 -186.18477045883455) / Math.Sqrt((591.2438750656617 / (299)) + (341.80711886433954 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17289244197671394 = (225.1034568028851 - 186.18477045883455) / 225.1034568028851 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)