search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 5/9/2022 3:10:19 PM #5361

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Benchstone.MDBenchF.MDSqMtx

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.MDBenchF.MDSqMtx.Test.html>) 931.90 ms 854.35 ms 0.92 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDSqMtx*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchF.MDSqMtx.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 854.3482563571428 < 885.4656778089285. IsChangePoint: Marked as a change because one of 3/17/2022 3:08:05 PM, 3/21/2022 1:28:14 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 15.349970071023957 (T) = (0 -843831903.5126032) / Math.Sqrt((1526553892445336.2 / (299)) + (759621418900283.4 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.07682954247216041 = (914058608.1711309 - 843831903.5126032) / 914058608.1711309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 706.85 ns 631.62 ns 0.89 0.09 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 58.87 ns 47.21 ns 0.80 0.07 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 726.82 ns 648.07 ns 0.89 0.09 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 711.15 ns 640.75 ns 0.90 0.11 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 114.16 ns 89.98 ns 0.79 0.08 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 59.51 ns 47.68 ns 0.80 0.09 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 701.28 ns 629.54 ns 0.90 0.09 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 631.6184433464551 < 668.3605285693475. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 9.833791966546812 (T) = (0 -553.922460970375) / Math.Sqrt((4711.711634962292 / (299)) + (1565.4716956360508 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.1099025201868542 = (622.3166265864021 - 553.922460970375) / 622.3166265864021 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 47.21085046039679 < 55.92003958745241. IsChangePoint: Marked as a change because one of 3/16/2022 12:48:00 PM, 4/14/2022 8:50:18 PM, 5/9/2022 5:23:46 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 52.344237281571736 (T) = (0 -47.51228669628864) / Math.Sqrt((6.675554312132443 / (299)) + (0.10898667420841743 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.14757952719851938 = (55.73808726125438 - 47.51228669628864) / 55.73808726125438 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 648.0718775044558 < 688.7298282358482. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 9.278116464256826 (T) = (0 -566.4950043635274) / Math.Sqrt((5223.502795113284 / (299)) + (1449.3054435504923 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.10158612447081776 = (630.5501504302251 - 566.4950043635274) / 630.5501504302251 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 640.7501596548083 < 691.361986157706. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 9.360318768952284 (T) = (0 -566.6133208387449) / Math.Sqrt((5360.645031355987 / (299)) + (1419.4293851061 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.10221679317404121 = (631.1248824111573 - 566.6133208387449) / 631.1248824111573 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 89.9846569548872 < 108.84136262090061. IsChangePoint: Marked as a change because one of 3/8/2022 5:44:17 PM, 4/14/2022 8:50:18 PM, 5/9/2022 5:23:46 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 45.21119838947206 (T) = (0 -90.9083883113855) / Math.Sqrt((36.30845269282431 / (299)) + (0.31484085746695406 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.15119130534860037 = (107.10115115953306 - 90.9083883113855) / 107.10115115953306 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 47.68091031965491 < 56.52465717810454. IsChangePoint: Marked as a change because one of 3/16/2022 3:05:50 AM, 4/14/2022 8:50:18 PM, 5/9/2022 5:23:46 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 45.94731542158267 (T) = (0 -47.57156745014671) / Math.Sqrt((9.21867270863119 / (299)) + (0.01579192334588221 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.14569044324134786 = (55.68422719118192 - 47.57156745014671) / 55.68422719118192 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 629.5390765572954 < 667.3202028941489. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 9.872145516738343 (T) = (0 -553.6621616900983) / Math.Sqrt((4783.92085844047 / (299)) + (1534.6767303554723 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.10992308556952798 = (622.0385594927677 - 553.6621616900983) / 622.0385594927677 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NextBytes_span_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_span_unseeded.html>) 169.83 ns 142.85 ns 0.84 0.04 False
[NextBytes_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_unseeded.html>) 169.66 ns 143.76 ns 0.85 0.04 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.NextBytes_span_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 142.85421545599647 < 161.2857049570197. IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 12.619414921499736 (T) = (0 -144.9704366738445) / Math.Sqrt((162.94877043354654 / (299)) + (42.45409261570316 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.09426368872902163 = (160.0581039645127 - 144.9704366738445) / 160.0581039645127 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 143.7640851528268 < 161.18623557429797. IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 12.660740228556143 (T) = (0 -145.18770764203546) / Math.Sqrt((157.94427380747956 / (299)) + (41.54513141068259 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.09334014254786335 = (160.1347037135148 - 145.18770764203546) / 160.1347037135148 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.AddGivenSize(Int32).Dictionary(Size%3a%20512).html>) 3.77 μs 3.26 μs 0.87 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.2645233376600906 < 3.6010273772622328. IsChangePoint: Marked as a change because one of 3/14/2022 4:00:26 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 16.56096469315896 (T) = (0 -3244.617235005693) / Math.Sqrt((56247.85479314987 / (299)) + (877.1015243365971 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.06831830199011492 = (3482.5383410840254 - 3244.617235005693) / 3482.5383410840254 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20Array).html>) 210.59 ns 190.53 ns 0.90 0.08 False
[Select - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20List).html>) 658.28 ns 606.19 ns 0.92 0.05 False
[SelectToList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.SelectToList(input%3a%20List).html>) 277.06 ns 230.96 ns 0.83 0.01 True
[Max - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Max(input%3a%20IEnumerable).html>) 346.34 ns 324.80 ns 0.94 0.02 False
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Min(input%3a%20IEnumerable).html>) 367.99 ns 346.26 ns 0.94 0.04 False
[All_AllElementsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input%3a%20IEnumerable).html>) 482.80 ns 455.97 ns 0.94 0.05 False
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20Array).html>) 211.73 ns 191.31 ns 0.90 0.07 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 190.5287769191199 < 200.66305368331535. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 12.080717701123499 (T) = (0 -190.11155310618096) / Math.Sqrt((145.56896233552777 / (299)) + (19.031865382505025 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.05635828176806389 = (201.46582058960408 - 190.11155310618096) / 201.46582058960408 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Select(input: List) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 606.186162436998 < 624.4593607734578. IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/20/2022 2:49:46 PM, 5/9/2022 3:10:19 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 20.583475269535082 (T) = (0 -610.6566857894366) / Math.Sqrt((413.58037030122017 / (299)) + (130.5346661701463 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.06433081285794988 = (652.6416538890777 - 610.6566857894366) / 652.6416538890777 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: List) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 230.9631374146358 < 263.4069135549628. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/13/2022 5:25:17 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 39.67231654444267 (T) = (0 -235.13891423180615) / Math.Sqrt((4.995376207415047 / (299)) + (20.979229287003474 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.14833478520375695 = (276.09312925626773 - 235.13891423180615) / 276.09312925626773 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 324.795765810557 < 328.90506019318417. IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 29.463509874050487 (T) = (0 -325.98114519724294) / Math.Sqrt((148.4826102684729 / (299)) + (17.345303405960557 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.07725173019403112 = (353.27201996898754 - 325.98114519724294) / 353.27201996898754 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 346.2613353507484 < 349.39922218863734. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/13/2022 5:30:39 PM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 30.03842045559216 (T) = (0 -347.4772468242042) / Math.Sqrt((223.31647992239118 / (299)) + (17.999457534007085 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.08388270082754325 = (379.2934017708059 - 347.4772468242042) / 379.2934017708059 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 455.965639198271 < 464.50432538594924. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 17.637363745261542 (T) = (0 -457.20121205764286) / Math.Sqrt((472.6262460434577 / (299)) + (25.406424732011512 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.053062515748852256 = (482.82090387329464 - 457.20121205764286) / 482.82090387329464 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 191.30580453735385 < 201.91577545740822. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 13.454387764250333 (T) = (0 -191.1696102097753) / Math.Sqrt((109.88662510399284 / (299)) + (18.919919830050798 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.05786853977173548 = (202.91182099306792 - 191.1696102097753) / 202.91182099306792 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Benchstone.BenchF.InvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchF.InvMt.Test.html>) 2.35 ms 1.92 ms 0.82 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.InvMt*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.InvMt.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.918404443981482 < 2.2350965747767852. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 18.44341269557009 (T) = (0 -1946407.6181448912) / Math.Sqrt((27074259206.08944 / (299)) + (11106155496.50001 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.14530991859590273 = (2277325.6183660217 - 1946407.6181448912) / 2277325.6183660217 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).Dictionary(Size%3a%20512).html>) 2.61 μs 2.24 μs 0.86 0.11 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.2353370024875616 < 2.4790051865917087. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 3/31/2022 1:23:38 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 7.508943189847423 (T) = (0 -2415.2276304104826) / Math.Sqrt((59140.68235564801 / (299)) + (14037.033239760553 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.06440366537183212 = (2581.484707686848 - 2415.2276304104826) / 2581.484707686848 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Benchstone.BenchI.TreeInsert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchI.TreeInsert.Test.html>) 51.08 μs 46.38 μs 0.91 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.TreeInsert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.TreeInsert.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 46.38325790854176 < 48.475332550247636. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/20/2022 8:23:28 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 24.53660078871692 (T) = (0 -46801.38656442428) / Math.Sqrt((2200592.3490415225 / (299)) + (541376.7153630425 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.06719402710092166 = (50172.69177529998 - 46801.38656442428) / 50172.69177529998 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryAddGiventSize(Int32).Dictionary(Count%3a%20512).html>) 4.59 μs 3.33 μs 0.73 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddGiventSize<Int32>.Dictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.334125357525237 < 4.413398894835184. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 13.643757965019741 (T) = (0 -3258.4587181639836) / Math.Sqrt((400148.4800556268 / (299)) + (1918.050463420724 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.13453494427327062 = (3764.980107056849 - 3258.4587181639836) / 3764.980107056849 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrue(Int32).Stack(Size%3a%20512).html>) 44.30 μs 37.17 μs 0.84 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 37.172897723214284 < 42.10569140976998. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 13.00157673243801 (T) = (0 -37351.51197337531) / Math.Sqrt((12598451.029841725 / (299)) + (2134759.481117951 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.09292132969388961 = (41177.80871285437 - 37351.51197337531) / 41177.80871285437 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%22%2c%20value%3a%20%22x%22%2c%20comparisonType%3a%20InvariantCultureIgno.html>) 240.67 ns 207.24 ns 0.86 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 207.2383250212224 < 228.58962038238957. IsChangePoint: Marked as a change because one of 3/15/2022 11:00:34 PM, 4/9/2022 5:43:54 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM. IsImprovementStdDev: Marked as improvement because 18.177310894659648 (T) = (0 -208.12968022904306) / Math.Sqrt((228.36380410556407 / (299)) + (48.41588377391404 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.104161882076913 = (232.32956497940873 - 208.12968022904306) / 232.32956497940873 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)