search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 21 Regressions on 1/3/2023 9:28:36 AM #11424

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22(%3f%3a(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%5c%5c.)%7b3%7d(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%22%2c%20Options%3a%20NonBacktracking).html>) 16.47 ms 17.66 ms 1.07 0.19 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22(%3f%3a(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%5c%5c.)%7b3%7d(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%22%2c%20Options%3a%20None).html>) 24.21 ms 25.75 ms 1.06 0.16 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.663875035714284 > 17.190590520999997. IsChangePoint: Marked as a change because one of 10/17/2022 12:47:34 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -12.307702614818494 (T) = (0 -17686218.04508533) / Math.Sqrt((192905613697.8846 / (14)) + (4256415541.6738687 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.08943440553510679 = (16234312.001921987 - 17686218.04508533) / 16234312.001921987 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.7492055 > 24.851100591250002. IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -12.632655449201454 (T) = (0 -25741007.92626581) / Math.Sqrt((319858367147.30347 / (14)) + (20895660430.107456 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.0813918302518762 = (23803590.15683543 - 25741007.92626581) / 23803590.15683543 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bLu%7d%22%2c%20Options%3a%20NonBacktracking).html>) 2.20 ms 2.38 ms 1.08 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-zA-Z%5d%2bing%22%2c%20Options%3a%20Compiled).html>) 3.20 ms 3.41 ms 1.07 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20NonBacktracking).html>) 3.69 ms 4.01 ms 1.09 0.02 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Lu}", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.382103474920635 > 2.2918389537355717. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 12/12/2022 5:25:39 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -52.3989559255251 (T) = (0 -2399576.671429496) / Math.Sqrt((102399779.63368474 / (14)) + (294568039.66992193 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.09835540854631576 = (2184699.6452681553 - 2399576.671429496) / 2184699.6452681553 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4076152942307694 > 3.237295330026042. IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -19.53720014673781 (T) = (0 -3419328.3394436813) / Math.Sqrt((8091215799.671299 / (27)) + (541808165.6707829 / (18))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (18) - 2, .025) and -0.11577270834608551 = (3064538.4260313786 - 3419328.3394436813) / 3064538.4260313786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.014150760368663 > 3.900485955100633. IsChangePoint: Marked as a change because one of 10/17/2022 12:47:34 PM, 11/10/2022 9:13:35 AM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -18.00059977348425 (T) = (0 -3960248.7309155003) / Math.Sqrt((333222229.9115126 / (14)) + (5067768351.181176 / (32))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (32) - 2, .025) and -0.06535783691269656 = (3717294.409164827 - 3960248.7309155003) / 3717294.409164827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyThreeValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Char).IndexOfAnyThreeValues(Size%3a%2033).html>) 4.38 ns 5.63 ns 1.29 0.07 False
[IndexOfValue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Char).IndexOfValue(Size%3a%20512).html>) 10.21 ns 11.49 ns 1.13 0.45 False
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Char).IndexOfAnyTwoValues(Size%3a%20512).html>) 12.39 ns 15.22 ns 1.23 0.16 False
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Char).IndexOfAnyTwoValues(Size%3a%2033).html>) 3.53 ns 4.83 ns 1.37 0.10 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.628166820876514 > 4.597105909855597. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -36.77680681536172 (T) = (0 -5.655804818241447) / Math.Sqrt((0.013263419071886317 / (14)) + (0.009863055027815103 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.3009369655422534 = (4.347485672285442 - 5.655804818241447) / 4.347485672285442 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.IndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.489458695578046 > 10.266903601736507. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -18.62737542350612 (T) = (0 -11.817693404526066) / Math.Sqrt((0.08959359098424977 / (14)) + (0.16646728659383475 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.20627319580679504 = (9.796863136482118 - 11.817693404526066) / 9.796863136482118 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.216317947785562 > 13.597551347474198. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -17.235833834876175 (T) = (0 -15.093266400070991) / Math.Sqrt((0.1918799707391884 / (14)) + (0.08203076526048839 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.1709970050517253 = (12.889244237993838 - 15.093266400070991) / 12.889244237993838 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.828465050501491 > 3.7061544722301516. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -97.2728698643746 (T) = (0 -4.8130289537697175) / Math.Sqrt((3.682342828588698E-05 / (14)) + (0.005288035035049038 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.3623758021019753 = (3.532820346885064 - 4.8130289537697175) / 3.532820346885064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in BenchmarksGame.RegexRedux_1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[RunBench - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/BenchmarksGame.RegexRedux_1.RunBench.html>) 31.43 ms 34.36 ms 1.09 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.RegexRedux_1*'
### Payloads [Baseline]() [Compare]() ### Histogram #### BenchmarksGame.RegexRedux_1.RunBench ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.356844075000005 > 33.202115351785714. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -34.904359065674946 (T) = (0 -34618915.406182796) / Math.Sqrt((49095962498.06905 / (14)) + (115525507873.77806 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.09379410499874363 = (31650303.515050083 - 34618915.406182796) / 31650303.515050083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5c%5cp%7bSm%7d%22%2c%20Options%3a%20None).html>) 31.84 ms 35.30 ms 1.11 0.13 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22(%3fi)Twain%22%2c%20Options%3a%20NonBacktracking).html>) 6.86 ms 7.41 ms 1.08 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5c%5cp%7bSm%7d%22%2c%20Options%3a%20NonBacktracking).html>) 28.49 ms 35.30 ms 1.24 0.15 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.30007569523809 > 33.438542860714286. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -708.9072782928235 (T) = (0 -35306031.012256235) / Math.Sqrt((179179868.22581518 / (14)) + (328212315.98487854 / (30))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (30) - 2, .025) and -0.10843838832445574 = (31852046.432301708 - 35306031.012256235) / 31852046.432301708 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.4085756020202025 > 7.291292092083333. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/4/2022 9:11:54 AM, 12/19/2022 2:44:31 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -16.47072415470621 (T) = (0 -7402012.87544073) / Math.Sqrt((8908944993.682022 / (14)) + (1820310451.9045384 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.06232090715237732 = (6967774.827365794 - 7402012.87544073) / 6967774.827365794 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.29579919047619 > 29.95823330892857. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -27.044616289694 (T) = (0 -35344074.63996304) / Math.Sqrt((827603452732.9766 / (14)) + (1096172590.3270335 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.22864881517513264 = (28766620.863037307 - 35344074.63996304) / 28766620.863037307 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Replace_Char - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.Replace_Char(text%3a%20%22This%20is%20a%20very%20nice%20sentence%22%2c%20oldChar%3a%20%27i%27%2c%20newChar%3a%20%27I%27).html>) 17.07 ns 19.08 ns 1.12 0.01 False
[Replace_Char - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.Replace_Char(text%3a%20%22Hello%22%2c%20oldChar%3a%20%27a%27%2c%20newChar%3a%20%27b%27).html>) 3.87 ns 5.25 ns 1.36 0.14 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.077025755890862 > 17.946853181321785. IsChangePoint: Marked as a change because one of 11/10/2022 9:13:35 AM, 12/25/2022 7:48:20 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -38.75210088538315 (T) = (0 -19.36002233628097) / Math.Sqrt((0.0077478549623265745 / (14)) + (0.0916227397872035 / (32))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (32) - 2, .025) and -0.13250367327353726 = (17.094887012878484 - 19.36002233628097) / 17.094887012878484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.254150501262543 > 4.057522829953935. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -60.59457935126962 (T) = (0 -5.1178940879577395) / Math.Sqrt((3.3449037555927136E-05 / (14)) + (0.01314234795315684 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.323567151834647 = (3.8667430518078594 - 5.1178940879577395) / 3.8667430518078594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbbb.html>) 8.98 ns 10.15 ns 1.13 0.14 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%99%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%22%2c%20value%3a%20%22%e3%81%99%22%2c%20comparisonType%3a%20Ordinal).html>) 7.51 ns 8.71 ns 1.16 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.149850494356581 > 9.419208208178945. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -11.250234159851368 (T) = (0 -10.320576618038045) / Math.Sqrt((8.766566868187124E-05 / (14)) + (0.4456102913519054 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.15038006933079093 = (8.971449430658009 - 10.320576618038045) / 8.971449430658009 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.707428376649865 > 7.897536792966942. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -8.233067509181069 (T) = (0 -8.484574598993131) / Math.Sqrt((0.0737051756663628 / (14)) + (0.19663414421679354 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.11675108672682381 = (7.597552131210597 - 8.484574598993131) / 7.597552131210597 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Byte).IndexOfAnyTwoValues(Size%3a%20512).html>) 6.71 ns 8.76 ns 1.31 0.13 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.764908038530468 > 7.336924366949196. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 12/14/2022 5:20:21 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -12.507493318753696 (T) = (0 -7.951201045126191) / Math.Sqrt((0.02737587906460868 / (14)) + (0.134843492843042 / (32))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (32) - 2, .025) and -0.1409708441241541 = (6.968803003226422 - 7.951201045126191) / 6.968803003226422 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20Compiled).html>) 2.37 μs 2.56 μs 1.08 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5570293018020647 > 2.4797178079474884. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -25.721242527750718 (T) = (0 -2522.879787305329) / Math.Sqrt((399.5059777685291 / (14)) + (324.2279960021517 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.06799377889876394 = (2362.260752030537 - 2522.879787305329) / 2362.260752030537 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>) 131.32 μs 139.41 μs 1.06 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 139.40952101769912 > 137.80986355851718. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -119.66955152496621 (T) = (0 -139284.20571444806) / Math.Sqrt((19753.253129697147 / (14)) + (94919.11414612409 / (31))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (31) - 2, .025) and -0.06096419732606761 = (131280.7784329424 - 139284.20571444806) / 131280.7784329424 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAny - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny.html>) 4.89 ns 6.67 ns 1.36 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.669456846777501 > 5.138558790769339. IsChangePoint: Marked as a change because one of 11/4/2022 4:27:34 PM, 1/3/2023 6:29:42 AM, 1/9/2023 6:56:09 PM falls between 1/1/2023 3:57:25 AM and 1/9/2023 6:56:09 PM. IsRegressionStdDev: Marked as regression because -58.50329257620621 (T) = (0 -6.832218472814834) / Math.Sqrt((0.006341276408552079 / (14)) + (0.019870946163708273 / (32))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (32) - 2, .025) and -0.39006714786732655 = (4.9150276540935325 - 6.832218472814834) / 4.9150276540935325 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)