search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/25/2022 3:15:54 PM #5083

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSetReverse(String).IList(Size%3a%20512).html>) 1.46 μs 1.79 μs 1.23 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSetReverse<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.793033342377949 > 1.52885829595421. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.826774173057736 (T) = (0 -1844.437788831213) / Math.Sqrt((12096.58161795882 / (20)) + (1876.2064688700889 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.23617008599814826 = (1492.058261013425 - 1844.437788831213) / 1492.058261013425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Range).html>) 199.22 ns 221.58 ns 1.11 0.15 False
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20Array).html>) 190.74 ns 211.64 ns 1.11 0.17 False
[All_AllElementsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input%3a%20IEnumerable).html>) 454.80 ns 494.77 ns 1.09 0.05 False
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20Array).html>) 191.26 ns 212.25 ns 1.11 0.14 False
[Concat_Once - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Concat_Once(input%3a%20IEnumerable).html>) 1.40 μs 1.59 μs 1.14 0.07 False
[WhereLast_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input%3a%20Array).html>) 192.16 ns 214.52 ns 1.12 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 221.58261138199498 > 207.99358039652964. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -11.970497096861765 (T) = (0 -219.87499754568375) / Math.Sqrt((52.10034154325395 / (20)) + (2.1189983093745535 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.0973539854894762 = (200.36834098489032 - 219.87499754568375) / 200.36834098489032 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 211.63954818097915 > 200.1543327733995. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.742329866370524 (T) = (0 -210.8693079539597) / Math.Sqrt((41.24672860016246 / (20)) + (0.5674762414903245 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.09536573806452706 = (192.5104105652952 - 210.8693079539597) / 192.5104105652952 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 494.76814783965307 > 477.9255502022633. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -19.427296800446392 (T) = (0 -489.09925490828834) / Math.Sqrt((45.71067362426424 / (20)) + (15.829213192428234 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.0693034068301376 = (457.3998846203839 - 489.09925490828834) / 457.3998846203839 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 212.2476370225824 > 199.8403765251347. IsChangePoint: Marked as a change because one of 3/1/2022 5:49:50 AM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.518522482869615 (T) = (0 -212.21186592224552) / Math.Sqrt((43.46004308152504 / (20)) + (0.5292884626811962 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.10396856113577398 = (192.22636712038232 - 212.21186592224552) / 192.22636712038232 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5862963145858102 > 1.471412613036221. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.556382187601544 (T) = (0 -1581.885941357156) / Math.Sqrt((3100.0534005666645 / (20)) + (188.31090204020666 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.11141032118690881 = (1423.3140643032827 - 1581.885941357156) / 1423.3140643032827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.52265695803766 > 202.3935561752828. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/7/2022 10:45:01 PM, 3/12/2022 1:22:26 AM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.848492263973935 (T) = (0 -213.97813486299665) / Math.Sqrt((43.660375986061894 / (20)) + (0.33672532345517087 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.09755590514431506 = (194.95875687066817 - 213.97813486299665) / 194.95875687066817 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 531.36 ns 704.38 ns 1.33 0.10 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 20.99 ns 23.12 ns 1.10 0.07 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 101.97 ns 114.82 ns 1.13 0.09 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 551.38 ns 717.15 ns 1.30 0.11 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 589.30 ns 636.25 ns 1.08 0.08 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 532.06 ns 704.45 ns 1.32 0.12 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 53.67 ns 59.47 ns 1.11 0.08 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 550.12 ns 721.33 ns 1.31 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 704.3769203451162 > 560.1227778412392. IsChangePoint: Marked as a change because one of 2/22/2022 2:43:53 PM, 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -24.056911209877974 (T) = (0 -703.9441142799851) / Math.Sqrt((886.5962448609621 / (20)) + (12.130557993018025 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.2957996728727976 = (543.2507269579222 - 703.9441142799851) / 543.2507269579222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.118002420027917 > 21.887179304795744. IsChangePoint: Marked as a change because one of 3/16/2022 12:48:00 PM, 3/17/2022 2:30:39 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -2.646234489974291 (T) = (0 -23.22906699676804) / Math.Sqrt((8.282735654888661 / (20)) + (0.022654393442385366 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.0791661282621302 = (21.52501490588452 - 23.22906699676804) / 21.52501490588452 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 114.81898051257306 > 107.25268318545304. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/8/2022 5:44:17 PM, 4/14/2022 8:50:18 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.42339866708789 (T) = (0 -114.5510036567658) / Math.Sqrt((13.899812084111003 / (20)) + (0.17182028929270035 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.10862871126986809 = (103.32675177206484 - 114.5510036567658) / 103.32675177206484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 717.1527018813789 > 578.2114389452263. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/7/2022 10:45:01 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -26.47062283312086 (T) = (0 -718.335440970908) / Math.Sqrt((709.1497347640772 / (20)) + (63.782309899451256 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.28878785884194164 = (557.3729113311002 - 718.335440970908) / 557.3729113311002 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 636.250840201839 > 618.8364670319796. IsChangePoint: Marked as a change because one of 2/23/2022 3:28:40 AM, 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -327.60737709043826 (T) = (0 -636.2388149069668) / Math.Sqrt((0.2235535228431929 / (20)) + (0.37477175105944915 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.07921093556243311 = (589.5407412410898 - 636.2388149069668) / 589.5407412410898 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 704.451313820622 > 558.7760289060137. IsChangePoint: Marked as a change because one of 2/22/2022 2:43:53 PM, 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -24.262750819516462 (T) = (0 -703.9764938841832) / Math.Sqrt((878.0781837079659 / (20)) + (10.045783235617094 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.2969971563651185 = (542.7741228493537 - 703.9764938841832) / 542.7741228493537 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.46963644945419 > 56.35861738018634. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.441361649156125 (T) = (0 -59.42915311506257) / Math.Sqrt((2.9472250895831595 / (20)) + (0.03380288483471498 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.0953690269116467 = (54.254914695388926 - 59.42915311506257) / 54.254914695388926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 721.3306622201887 > 575.9710914546783. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/7/2022 10:45:01 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -25.725037273910388 (T) = (0 -714.274125857387) / Math.Sqrt((701.2179788256748 / (20)) + (80.04620054901204 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.28037061288593296 = (557.8651358198746 - 714.274125857387) / 557.8651358198746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Log2_ulong - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_BitOperations.Log2_ulong.html>) 482.07 ns 528.55 ns 1.10 0.01 False
[Log2_uint - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_BitOperations.Log2_uint.html>) 484.08 ns 520.34 ns 1.07 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_BitOperations.Log2_ulong ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 528.5458231173697 > 507.7006415909637. IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -16.747685074151324 (T) = (0 -526.4300131119818) / Math.Sqrt((112.47872185933909 / (20)) + (4.4446747568591825 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.08242937218894614 = (486.3412123115313 - 526.4300131119818) / 486.3412123115313 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BitOperations.Log2_uint ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 520.3369046028087 > 508.5691699078917. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/21/2022 3:06:05 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -59.20072940377172 (T) = (0 -515.9899256793219) / Math.Sqrt((2.7515294259299155 / (20)) + (6.154252490352192 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.06513763586219282 = (484.434976576192 - 515.9899256793219) / 484.434976576192 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%2212345%22).html>) 12.27 ns 13.58 ns 1.11 0.10 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%22-2147483648%22).html>) 16.97 ns 20.72 ns 1.22 0.14 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%222147483647%22).html>) 16.66 ns 19.59 ns 1.18 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int32.TryParseSpan(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.584426540473862 > 12.712152793742126. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/6/2022 9:32:10 PM, 3/31/2022 1:45:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -11.310343892977603 (T) = (0 -13.527124874144208) / Math.Sqrt((0.2890818314340876 / (20)) + (0.013767315558523074 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.11319402381248814 = (12.151632675691388 - 13.527124874144208) / 12.151632675691388 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.719345162467313 > 17.591341146626775. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/31/2022 1:45:25 PM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.385821601617371 (T) = (0 -20.61498274223717) / Math.Sqrt((1.558000508101929 / (20)) + (0.1379227977615555 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.20654823771321829 = (17.085916748184832 - 20.61498274223717) / 17.085916748184832 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParseSpan(value: "2147483647") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.58797024370232 > 17.100513029940927. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/4/2022 7:13:54 PM, 3/31/2022 1:45:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -10.933323230932569 (T) = (0 -18.991208018356765) / Math.Sqrt((0.842095082261733 / (20)) + (0.1122965679112246 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1387244186950788 = (16.677615502546033 - 18.991208018356765) / 16.677615502546033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(String).ImmutableArray(Size%3a%20512).html>) 664.34 μs 769.97 μs 1.16 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 769.9693937246965 > 696.7153744140626. IsChangePoint: Marked as a change because one of 2/23/2022 10:45:30 PM, 3/3/2022 7:35:51 PM, 3/11/2022 8:42:43 PM, 3/30/2022 6:37:35 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.199640924281566 (T) = (0 -769504.3015730736) / Math.Sqrt((1056731447.6362686 / (20)) + (1994333.9279473082 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1425205410495983 = (673514.6318386134 - 769504.3015730736) / 673514.6318386134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrue(String).ImmutableArray(Size%3a%20512).html>) 336.82 μs 388.78 μs 1.15 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 388.7828114837399 > 352.7697491356383. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -14.081796644745339 (T) = (0 -389756.7055776589) / Math.Sqrt((239352050.54557398 / (20)) + (353014.14645937446 / (43))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (43) - 2, .025) and -0.14289735417916008 = (341025.11844345456 - 389756.7055776589) / 341025.11844345456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.AddGivenSize(Int32).Dictionary(Size%3a%20512).html>) 3.23 μs 3.78 μs 1.17 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.779988598096633 > 3.4159420819099817. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -62.34906045246143 (T) = (0 -3773.7169896737773) / Math.Sqrt((1135.6095713538443 / (20)) + (591.6442106544841 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1615561829119141 = (3248.84585454439 - 3773.7169896737773) / 3248.84585454439 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Byte).DotBenchmark.html>) 70.37 ns 81.45 ns 1.16 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.DotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 81.45299846756448 > 75.63286282246523. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.129590186332507 (T) = (0 -81.78113135799418) / Math.Sqrt((9.986433691128417 / (20)) + (0.16247348514989948 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.11758127785494295 = (73.17689816257732 - 81.78113135799418) / 73.17689816257732 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArraySet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Tests.Perf_BitArray.BitArraySet(Size%3a%20512).html>) 614.03 ns 783.63 ns 1.28 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArraySet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 783.6326430413882 > 644.1761298927876. IsChangePoint: Marked as a change because one of 3/2/2022 3:17:47 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.055945472656125 (T) = (0 -783.4227763239863) / Math.Sqrt((2732.4741728878753 / (20)) + (0.230028001375361 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.24192395779943898 = (630.813804181804 - 783.4227763239863) / 630.813804181804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Double.Min.html>) 4.42 μs 6.50 μs 1.47 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Double.Min ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.4975044014206516 > 4.66380701464377. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.958249915380495 (T) = (0 -6495.3177851838045) / Math.Sqrt((405997.2549599571 / (20)) + (35.806928811562365 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.3971373337661743 = (4649.018838881636 - 6495.3177851838045) / 4649.018838881636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test0 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Burgers.Test0.html>) 279.43 ms 302.74 ms 1.08 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Burgers*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Burgers.Test0 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 302.7422366666667 > 293.7870055125. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.675114612954975 (T) = (0 -301623583.98660773) / Math.Sqrt((45609583406813.61 / (20)) + (706022800636.1838 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.06803270532392441 = (282410437.8855403 - 301623583.98660773) / 282410437.8855403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in Benchstone.BenchF.InvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchF.InvMt.Test.html>) 1.92 ms 2.35 ms 1.23 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.InvMt*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.InvMt.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.351694253365385 > 2.0126675343149034. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.111417261717229 (T) = (0 -2352450.209598353) / Math.Sqrt((17804707352.346935 / (20)) + (35607034.78812777 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.199583060774417 = (1961056.5424954214 - 2352450.209598353) / 1961056.5424954214 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormatL - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Version.TryFormatL.html>) 55.80 ns 59.84 ns 1.07 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Version.TryFormatL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.83523464742666 > 58.60627234433506. IsChangePoint: Marked as a change because one of 2/22/2022 2:43:53 PM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -14.376985454944126 (T) = (0 -59.85091567072055) / Math.Sqrt((1.2494140083037244 / (20)) + (0.006449673766319912 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.0639576806132549 = (56.25309799561113 - 59.85091567072055) / 56.25309799561113 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IEnumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(String).IEnumerable(Size%3a%20512).html>) 1.80 μs 2.03 μs 1.13 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.IEnumerable(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.027557863609813 > 1.8933113981570737. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.382342044486345 (T) = (0 -2026.4144848716585) / Math.Sqrt((4527.309088159281 / (20)) + (4.585167888606164 / (44))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (44) - 2, .025) and -0.11034899024953629 = (1825.0248369355013 - 2026.4144848716585) / 1825.0248369355013 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.50).html>) 3.70 ns 4.76 ns 1.29 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'
### Payloads [Baseline]() [Compare]() ### Histogram #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.50) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.758761956498061 > 3.880668604601569. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 3/31/2022 1:45:25 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.12931658956812 (T) = (0 -4.769735403625637) / Math.Sqrt((0.10748022731625724 / (20)) + (5.19662971991152E-05 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.252837769525967 = (3.807145282210281 - 4.769735403625637) / 3.807145282210281 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Max - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.Max.html>) 4.35 μs 5.44 μs 1.25 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Single.Max ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.443945799965181 > 4.573442311240312. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.986643727618018 (T) = (0 -5443.489004237523) / Math.Sqrt((113440.44326200135 / (20)) + (9.58531520694941 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.2190350884979625 = (4465.40797356763 - 5443.489004237523) / 4465.40797356763 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in Benchstone.MDBenchF.MDSqMtx

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.MDBenchF.MDSqMtx.Test.html>) 823.40 ms 931.63 ms 1.13 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDSqMtx*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchF.MDSqMtx.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 931.6267643333334 > 864.4936331625001. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.032628371152452 (T) = (0 -931758537.429326) / Math.Sqrt((1115011543227623.1 / (20)) + (261483883239.62973 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.11662297682903743 = (834443278.3170148 - 931758537.429326) / 834443278.3170148 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NextBytes_span_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_span_unseeded.html>) 143.06 ns 169.69 ns 1.19 0.07 False
[NextBytes_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_unseeded.html>) 143.53 ns 169.67 ns 1.18 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.NextBytes_span_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.68715385376143 > 150.43968033751437. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.003761480290883 (T) = (0 -169.79751097603878) / Math.Sqrt((67.24947190052598 / (20)) + (0.02370504068760032 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1633914952490858 = (145.95044889827443 - 169.79751097603878) / 145.95044889827443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.66636452986896 > 150.72569522925525. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.83458975815315 (T) = (0 -169.70461698050087) / Math.Sqrt((67.02750460664382 / (20)) + (0.009146543950689128 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.1607079127541241 = (146.20785739094885 - 169.70461698050087) / 146.20785739094885 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsciiCharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars%3a%202000000).html>) 117.27 μs 140.10 μs 1.19 0.30 False
[WriteAsciiString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars%3a%202000000).html>) 201.60 μs 226.38 μs 1.12 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 140.09614520089283 > 122.81467059235075. IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/7/2022 3:14:09 PM, 4/21/2022 7:46:01 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -103.25771206206763 (T) = (0 -139842.21093236769) / Math.Sqrt((889236.4046022388 / (20)) + (111311.30876319151 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1908580049157851 = (117429.79461456198 - 139842.21093236769) / 117429.79461456198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 226.38175966880343 > 214.31750993215462. IsChangePoint: Marked as a change because one of 3/30/2022 7:10:16 AM, 4/7/2022 3:14:09 PM, 4/19/2022 11:38:14 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -7.852674729353239 (T) = (0 -226443.44048603452) / Math.Sqrt((76667170.66233821 / (20)) + (54176443.4129446 / (41))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (41) - 2, .025) and -0.08546255884922131 = (208614.6948505565 - 226443.44048603452) / 208614.6948505565 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare_Same_Upper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20OrdinalIgnoreCase)).html>) 1.19 μs 1.54 μs 1.29 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5362156851452469 > 1.2530088212507628. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -23.720478202770558 (T) = (0 -1536.530300944289) / Math.Sqrt((3699.5196009856168 / (20)) + (26.429585397098883 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.2663333025410635 = (1213.3695748670905 - 1536.530300944289) / 1213.3695748670905 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%22%2c%20value%3a%20%22x%22%2c%20comparisonType%3a%20InvariantCultureIgno.html>) 205.63 ns 241.41 ns 1.17 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 241.4145068795985 > 216.60573343122795. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/15/2022 11:00:34 PM, 4/9/2022 5:43:54 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.859872461967779 (T) = (0 -240.81865559794815) / Math.Sqrt((114.84712138828014 / (20)) + (1.3780997945061713 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1472233990464966 = (209.9143512920868 - 240.81865559794815) / 209.9143512920868 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 117.75 ns 227.19 ns 1.93 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 227.19138502966928 > 123.51455944604889. IsChangePoint: Marked as a change because one of 2/23/2022 10:45:30 PM, 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.078006442717768 (T) = (0 -227.16239870593648) / Math.Sqrt((1136.4033704597477 / (20)) + (0.04333138090688458 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.766692665351212 = (128.58059761106068 - 227.16239870593648) / 128.58059761106068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ClassVirtualMethod - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.ClassVirtualMethod.html>) 108.74 μs 130.55 μs 1.20 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ClassVirtualMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.54615614583335 > 114.24181926323784. IsChangePoint: Marked as a change because one of 3/16/2022 10:57:58 PM, 3/21/2022 1:28:14 PM, 4/20/2022 8:23:28 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.691755923050687 (T) = (0 -130722.2451882922) / Math.Sqrt((46000726.91008026 / (20)) + (1133648.9196731155 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.1738551340226613 = (111361.48013453984 - 130722.2451882922) / 111361.48013453984 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrue(Int32).Stack(Size%3a%20512).html>) 37.18 μs 44.30 μs 1.19 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.29837101628895 > 38.961584079902245. IsChangePoint: Marked as a change because one of 2/23/2022 10:45:30 PM, 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -12.978056093738894 (T) = (0 -44322.98833624293) / Math.Sqrt((4987541.170319388 / (20)) + (1628.3470538108995 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.17127847213784175 = (37841.546131505085 - 44322.98833624293) / 37841.546131505085 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(Int32).Stack(Size%3a%20512).html>) 66.20 μs 80.43 μs 1.21 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.42961514957265 > 69.50199961366035. IsChangePoint: Marked as a change because one of 2/23/2022 10:45:30 PM, 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -13.110286137753208 (T) = (0 -80467.01533175577) / Math.Sqrt((19219063.147549734 / (20)) + (2971.6877498793765 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.19008057705450573 = (67614.76229694855 - 80467.01533175577) / 67614.76229694855 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueFalse(Int32%2c%20Int32).Dictionary(Size%3a%20512).html>) 4.43 μs 5.63 μs 1.27 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.6268380768063135 > 4.6439803664021815. IsChangePoint: Marked as a change because one of 3/15/2022 11:34:41 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -326.49295320737565 (T) = (0 -5631.19131703942) / Math.Sqrt((140.88420043185897 / (20)) + (278.3815733013622 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.27288860113960467 = (4423.946692583993 - 5631.19131703942) / 4423.946692583993 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryAddGiventSize(Int32).Dictionary(Count%3a%20512).html>) 3.39 μs 4.64 μs 1.37 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddGiventSize<Int32>.Dictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.636764791913507 > 3.5300292363509085. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -80.74829963908546 (T) = (0 -4646.103916431913) / Math.Sqrt((4979.894037081333 / (20)) + (564.9781371721639 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.39189708895112113 = (3337.9651077027784 - 4646.103916431913) / 3337.9651077027784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in Benchstone.BenchI.TreeInsert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchI.TreeInsert.Test.html>) 46.99 μs 51.00 μs 1.09 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.TreeInsert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.TreeInsert.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.00037317538126 > 49.08135955177238. IsChangePoint: Marked as a change because one of 2/23/2022 5:12:02 AM, 2/26/2022 10:27:55 AM, 3/21/2022 1:28:14 PM, 4/20/2022 8:23:28 AM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsRegressionStdDev: Marked as regression because -16.371607531209115 (T) = (0 -51075.74271397043) / Math.Sqrt((1059333.396767634 / (20)) + (66415.33641909315 / (42))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (42) - 2, .025) and -0.08092074085508931 = (47252.07018746415 - 51075.74271397043) / 47252.07018746415 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
DrewScoggins commented 2 years ago

See if we can repro and get some ETW traces on this. https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu%2018.04/System.Numerics.Tests.Perf_BitOperations.Log2_uint.html