search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Linux/x64: 32 Improvements on 10/6/2022 5:26:59 PM #8351

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(String).ICollection(Size%3a%20512).html>) 763.23 μs 638.53 μs 0.84 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.ContainsFalse<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 638.5272358333334 < 726.9531525857426. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/6/2022 4:25:34 AM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/26/2022 8:26:31 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 16.515909260367412 (T) = (0 -638774.5460608467) / Math.Sqrt((2801752802.558781 / (299)) + (2218365.5751129608 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.07422936444515266 = (689992.2308272462 - 638774.5460608467) / 689992.2308272462 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WhereLast_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input%3a%20Array).html>) 214.83 ns 194.52 ns 0.91 0.04 False
[OrderByDescending - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.OrderByDescending(input%3a%20IEnumerable).html>) 4.58 μs 3.45 μs 0.75 0.12 False
[FirstWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input%3a%20IOrderedEnumerable).html>) 3.11 μs 2.15 μs 0.69 0.01 False
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20Array).html>) 212.58 ns 191.87 ns 0.90 0.04 False
[OrderBy - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.OrderBy(input%3a%20IEnumerable).html>) 3.52 μs 2.84 μs 0.81 0.24 False
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20IOrderedEnumerable).html>) 3.13 μs 2.13 μs 0.68 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 194.51713739565457 < 204.20692285433347. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 19.745060187363684 (T) = (0 -194.21953511245306) / Math.Sqrt((115.27732017249093 / (299)) + (0.46131124842906807 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.06296001622540696 = (207.2692077984721 - 194.21953511245306) / 207.2692077984721 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.4481201499118166 < 4.35330545252237. IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 82.68448963256951 (T) = (0 -3444.4821283454894) / Math.Sqrt((9210.984784402668 / (299)) + (4928.887876481444 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.24605422332716578 = (4568.607232665992 - 3444.4821283454894) / 4568.607232665992 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.1519805659601907 < 2.9471814923821285. IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 137.74762708195962 (T) = (0 -2145.7093487304314) / Math.Sqrt((2079.805533591089 / (299)) + (1201.351529795197 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2998360003413095 = (3064.581083541002 - 2145.7093487304314) / 3064.581083541002 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 191.86680955558316 < 202.02963334715844. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/15/2022 9:13:40 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 19.296874735151587 (T) = (0 -191.85950544891807) / Math.Sqrt((100.2667337709095 / (299)) + (1.1693808489549624 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.06420067409339027 = (205.02205989841153 - 191.85950544891807) / 205.02205989841153 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.836625351774746 < 3.3367983697035246. IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 67.61218013686015 (T) = (0 -2745.3121919434057) / Math.Sqrt((8167.441641565265 / (299)) + (3686.466246563232 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22504091426900705 = (3542.5253313261624 - 2745.3121919434057) / 3542.5253313261624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.126331613279833 < 2.9696117727937668. IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 151.513728213256 (T) = (0 -2187.742264766284) / Math.Sqrt((2007.5741670029684 / (299)) + (872.9609075298526 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2876443749606497 = (3071.1377686467117 - 2187.742264766284) / 3071.1377686467117 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ServiceProvider - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider.html>) 23.52 ns 21.12 ns 0.90 0.22 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.116655594071172 < 21.890394296637364. IsChangePoint: Marked as a change because one of 8/9/2022 6:41:10 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 6.738944132525363 (T) = (0 -21.741692006605078) / Math.Sqrt((1.2624257298185895 / (299)) + (0.9018758680855073 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09104474543866285 = (23.91943046426157 - 21.741692006605078) / 23.91943046426157 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 7.07 ns 6.01 ns 0.85 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.008967685876951 < 6.719125265116694. IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 6.2380702615328145 (T) = (0 -6.246443689687864) / Math.Sqrt((0.1106195855628779 / (299)) + (0.05373817822251411 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.07366482244313091 = (6.74317875540723 - 6.246443689687864) / 6.74317875540723 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Reflection.Activator<EmptyStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateInstanceType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Reflection.Activator(EmptyStruct).CreateInstanceType.html>) 9.14 ns 7.97 ns 0.87 0.16 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Activator&lt;EmptyStruct&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Reflection.Activator<EmptyStruct>.CreateInstanceType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.96606666366933 < 8.741319024776214. IsChangePoint: Marked as a change because one of 8/9/2022 6:41:10 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 14.579149092709699 (T) = (0 -8.260902093723994) / Math.Sqrt((0.7443937342742006 / (299)) + (0.08904912341869313 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.16415691909485838 = (9.883316955591942 - 8.260902093723994) / 9.883316955591942 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueTrue(Int32%2c%20Int32).IDictionary(Size%3a%20512).html>) 4.26 μs 3.73 μs 0.88 0.41 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.TryGetValueTrue<Int32, Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.729945431958912 < 4.085358326709172. IsChangePoint: Marked as a change because one of 8/5/2022 4:34:47 AM, 10/3/2022 2:18:04 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 14.527664427338975 (T) = (0 -3657.01649185987) / Math.Sqrt((180588.05594714882 / (299)) + (3569.816295237396 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.11163373613471532 = (4116.563900060971 - 3657.01649185987) / 4116.563900060971 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Base64DecodeDestinationTooSmall - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Base64Tests.Base64DecodeDestinationTooSmall(NumberOfBytes%3a%201000).html>) 58.61 ns 53.61 ns 0.91 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Text.Tests.Base64Tests.Base64DecodeDestinationTooSmall(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 53.60980292932157 < 55.68653650126676. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/22/2022 9:36:10 PM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 11.177272785084085 (T) = (0 -53.28489985296302) / Math.Sqrt((17.497256305557485 / (299)) + (0.5740586980699851 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.06834494164649633 = (57.193807273619505 - 53.28489985296302) / 57.193807273619505 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).ImmutableSortedSet(Size%3a%20512).html>) 16.40 μs 13.90 μs 0.85 0.08 False
[SortedList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).SortedList(Size%3a%20512).html>) 7.90 μs 6.26 μs 0.79 0.23 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).SortedSet(Size%3a%20512).html>) 11.38 μs 8.22 μs 0.72 0.13 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.89613441183927 < 15.534677831199733. IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/1/2022 8:04:48 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 14.65216486884287 (T) = (0 -13874.58866885333) / Math.Sqrt((2519425.282756625 / (299)) + (3841.8995952282135 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09038298790044756 = (15253.22029414159 - 13874.58866885333) / 15253.22029414159 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.256582906004264 < 7.508995841626271. IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/9/2022 4:43:10 PM, 10/3/2022 2:18:04 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 9.48615227172529 (T) = (0 -6759.305402072705) / Math.Sqrt((335432.04205221153 / (299)) + (219029.75561296742 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.11131879823260642 = (7605.995703104687 - 6759.305402072705) / 7605.995703104687 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.215061305298931 < 10.867445856368862. IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/22/2022 9:36:10 PM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 17.887773384598294 (T) = (0 -8283.733085851358) / Math.Sqrt((2827682.0599077228 / (299)) + (6092.25325131619 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.17856450152785244 = (10084.45958478654 - 8283.733085851358) / 10084.45958478654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BinarySearch - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Byte).BinarySearch(Size%3a%20512).html>) 11.20 ns 9.83 ns 0.88 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Memory.Span<Byte>.BinarySearch(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.834190158769418 < 10.635560224515308. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/4/2022 6:25:19 PM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 16.243454012831396 (T) = (0 -9.890588095366702) / Math.Sqrt((0.48620849079376105 / (299)) + (0.001494388069723974 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06472367533661229 = (10.575043796737175 - 9.890588095366702) / 10.575043796737175 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DelegateInvoke - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.DelegatePerf.DelegateInvoke.html>) 348.28 μs 304.61 μs 0.87 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.DelegatePerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.DelegatePerf.DelegateInvoke ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 304.60963846153845 < 330.6690861111111. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 23.859572044484963 (T) = (0 -304474.5059645861) / Math.Sqrt((451917169.4534018 / (299)) + (7255.633178568542 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.08789536941949798 = (333815.32749242336 - 304474.5059645861) / 333815.32749242336 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)