search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Alpine/x64: 17 Improvements on 10/6/2022 5:26:59 PM #8307

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Reflection.Activator<EmptyStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateInstanceType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Reflection.Activator(EmptyStruct).CreateInstanceType.html>) 8.76 ns 7.66 ns 0.87 0.22 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Activator&lt;EmptyStruct&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Reflection.Activator<EmptyStruct>.CreateInstanceType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.664239852794308 < 8.507569505620372. IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 11.918805542323552 (T) = (0 -7.73731622848168) / Math.Sqrt((0.5427793796929938 / (299)) + (0.06112894308249871 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.12061687833244704 = (8.798572587804044 - 7.73731622848168) / 8.798572587804044 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetConverter - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.SomeValueType%3f)).html>) 153.72 ns 133.10 ns 0.87 0.02 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(int)).html>) 152.39 ns 132.86 ns 0.87 0.02 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.Guid)).html>) 153.47 ns 133.96 ns 0.87 0.04 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.DerivedClass)).html>) 154.03 ns 133.26 ns 0.87 0.02 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 133.10236785795314 < 146.43467662652154. IsChangePoint: Marked as a change because one of 7/22/2022 1:09:02 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 22.93081004094185 (T) = (0 -132.63471117455478) / Math.Sqrt((157.9876211248752 / (299)) + (1.6490261291376556 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1258378904804128 = (151.72781996630667 - 132.63471117455478) / 151.72781996630667 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 132.85782554569414 < 144.61812697192113. IsChangePoint: Marked as a change because one of 7/22/2022 1:09:02 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 26.426994261686374 (T) = (0 -131.52360154113546) / Math.Sqrt((68.90436538941127 / (299)) + (3.1458881794172204 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.12917758160755077 = (151.0337799799986 - 131.52360154113546) / 151.0337799799986 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 133.96319887805802 < 144.81937475585497. IsChangePoint: Marked as a change because one of 7/22/2022 1:09:02 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 28.762911282465318 (T) = (0 -130.58534253366182) / Math.Sqrt((81.88912001181555 / (299)) + (2.343382206181355 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.13571262880478455 = (151.09018931177542 - 130.58534253366182) / 151.09018931177542 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 133.2568382256585 < 146.40086270026285. IsChangePoint: Marked as a change because one of 7/22/2022 1:09:02 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 30.95873511693611 (T) = (0 -131.35893131317795) / Math.Sqrt((100.44247966964625 / (299)) + (0.9949646098663529 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.13458708802502997 = (151.7875796576712 - 131.35893131317795) / 151.7875796576712 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.CtorFromCollection(Int32).ImmutableSortedSet(Size%3a%20512).html>) 15.06 μs 11.69 μs 0.78 0.14 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.CtorFromCollection(Int32).SortedSet(Size%3a%20512).html>) 10.73 μs 7.68 μs 0.72 0.10 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.685885635112836 < 14.2400516899099. IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 14.653300049509436 (T) = (0 -11755.54545128142) / Math.Sqrt((2344754.6830373947 / (299)) + (5152.3693708326 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10229562680120473 = (13095.11884117576 - 11755.54545128142) / 13095.11884117576 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.684460899770119 < 10.214456840842038. IsChangePoint: Marked as a change because one of 7/10/2022 8:13:50 PM, 8/1/2022 8:26:50 AM, 8/4/2022 6:25:19 PM, 9/9/2022 3:12:19 AM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 20.00961007118433 (T) = (0 -7736.375324674593) / Math.Sqrt((1985979.6339287625 / (299)) + (5717.282676142391 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.18011031459677238 = (9435.87346250098 - 7736.375324674593) / 9435.87346250098 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.CopyTo<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.CopyTo(String).ImmutableArray(Size%3a%202048).html>) 537.08 ns 369.01 ns 0.69 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CopyTo&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.CopyTo<String>.ImmutableArray(Size: 2048) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 369.007620558218 < 509.9993806044027. IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 41.64851437483297 (T) = (0 -374.6971506249285) / Math.Sqrt((2306.02652821624 / (299)) + (49.91314619482296 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.28375670853809537 = (523.1422829247648 - 374.6971506249285) / 523.1422829247648 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in PerfLabTests.EnumPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ObjectGetType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/PerfLabTests.EnumPerf.ObjectGetType.html>) 1.45 ns 0.23 ns 0.16 0.50 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.EnumPerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.EnumPerf.ObjectGetType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 0.2297298310307919 < 1.5087257612002098. IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 34.19346539750081 (T) = (0 -0.16456761713238466) / Math.Sqrt((0.18428765888813892 / (299)) + (0.0012295663580251062 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.8496127681841108 = (1.0942924817836641 - 0.16456761713238466) / 1.0942924817836641 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[StructWithInterfaceInterfaceMethod - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod.html>) 130.44 μs 108.71 μs 0.83 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 108.71073689569978 < 123.93716142165799. IsChangePoint: Marked as a change because one of 7/7/2022 8:07:51 AM, 7/13/2022 7:22:00 AM, 8/5/2022 10:47:59 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 15.684785169175854 (T) = (0 -108704.83563895884) / Math.Sqrt((117474806.53923221 / (299)) + (1421.1159618216052 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.0829537904815373 = (118538.01314553093 - 108704.83563895884) / 118538.01314553093 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsTrue(Int32).ImmutableHashSet(Size%3a%20512).html>) 16.81 μs 15.76 μs 0.94 0.01 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsTrue(Int32).SortedSet(Size%3a%20512).html>) 24.36 μs 22.80 μs 0.94 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.ContainsTrue<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.757577289937382 < 15.98780836075911. IsChangePoint: Marked as a change because one of 7/7/2022 8:07:51 AM, 7/13/2022 7:22:00 AM, 7/28/2022 3:42:13 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 8.63848054598823 (T) = (0 -15500.644991845973) / Math.Sqrt((197783.6242042724 / (299)) + (87960.95230403922 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.051410956155596585 = (16340.737954368084 - 15500.644991845973) / 16340.737954368084 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.804815784671536 < 23.411342291412076. IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 26.059090278425487 (T) = (0 -22879.138178735266) / Math.Sqrt((682876.1945835896 / (299)) + (5815.246508931831 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.05327581777887909 = (24166.63544503347 - 22879.138178735266) / 24166.63544503347 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsFalse(String).ICollection(Size%3a%20512).html>) 758.71 μs 631.89 μs 0.83 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.ContainsFalse<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 631.894276923077 < 721.5790801564408. IsChangePoint: Marked as a change because one of 7/7/2022 8:07:51 AM, 7/10/2022 1:04:01 AM, 7/13/2022 7:22:00 AM, 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 22.589256586282367 (T) = (0 -633577.3179196885) / Math.Sqrt((2025719912.8301928 / (299)) + (1841513.2935573459 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.08596881492757064 = (693168.1634795456 - 633577.3179196885) / 693168.1634795456 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.ContainsTrue(String).ImmutableArray(Size%3a%20512).html>) 384.11 μs 320.98 μs 0.84 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 320.9797927721088 < 365.8574394866071. IsChangePoint: Marked as a change because one of 7/6/2022 7:05:00 PM, 7/10/2022 1:04:01 AM, 7/13/2022 7:22:00 AM, 7/28/2022 3:42:13 AM, 8/4/2022 6:25:19 PM, 9/26/2022 8:26:31 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 6.757116685381473 (T) = (0 -326828.4417584223) / Math.Sqrt((384062263.95104355 / (299)) + (135191643.20149833 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.0736858863702521 = (352826.797032974 - 326828.4417584223) / 352826.797032974 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Ctor0_ActivatorCreateInstance_NoParams - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams.html>) 9.80 ns 8.19 ns 0.84 0.29 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Invoke*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.18979957078715 < 8.900138596170315. IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 15.652731937580471 (T) = (0 -8.219206615565383) / Math.Sqrt((0.5437370345636539 / (163)) + (0.038137503044448386 / (10))) is greater than 1.9739339541028664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (163) + (10) - 2, .975) and 0.13869392201738828 = (9.542724503717382 - 8.219206615565383) / 9.542724503717382 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentBag - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Collections.IterateForEach(String).ConcurrentBag(Size%3a%20512).html>) 4.14 μs 3.71 μs 0.89 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IterateForEach<String>.ConcurrentBag(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.7081355141771803 < 3.8846942104563946. IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 7/28/2022 6:38:38 AM, 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 17.580670479464963 (T) = (0 -3658.6601672760326) / Math.Sqrt((20230.24337168158 / (299)) + (2774.2069436397214 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.08194841788025245 = (3985.244662209851 - 3658.6601672760326) / 3985.244662209851 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS alpine 3.15
Baseline 656eafb92253b44ec375d933cc16fa3a46885a8f
Compare be5a4d36c04e6d8f4d0ecd58404bb0e49246f8d3
Diff Diff

Improvements in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DelegateInvoke - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/PerfLabTests.DelegatePerf.DelegateInvoke.html>) 347.98 μs 304.41 μs 0.87 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.DelegatePerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.DelegatePerf.DelegateInvoke ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 304.4081892566568 < 330.5282785596001. IsChangePoint: Marked as a change because one of 7/7/2022 8:07:51 AM, 7/13/2022 7:22:00 AM, 7/28/2022 6:38:38 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsImprovementStdDev: Marked as improvement because 21.473484939951714 (T) = (0 -304497.47413736704) / Math.Sqrt((451854082.19633514 / (299)) + (14601.825093105399 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.07981207317105266 = (330907.9213706851 - 304497.47413736704) / 330907.9213706851 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)