dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 5/9/2022 7:27:05 AM #5251

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Char).ToArray(Size%3a%20512).html>) 48.98 ns 45.23 ns 0.92 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 45.23203554067478 < 48.41345463162877. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/2/2022 3:40:16 AM, 4/8/2022 7:02:49 PM, 5/2/2022 11:50:30 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 48.26457451684611 (T) = (0 -45.47249653703938) / Math.Sqrt((4.098486107629955 / (299)) + (0.06409569192965535 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13829675411740447 = (52.77048305703478 - 45.47249653703938) / 52.77048305703478 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.CtorGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).Array(Size%3a%20512).html>) 61.44 ns 49.66 ns 0.81 0.03 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).Stack(Size%3a%20512).html>) 72.86 ns 58.36 ns 0.80 0.02 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).Queue(Size%3a%20512).html>) 75.08 ns 58.06 ns 0.77 0.02 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).HashSet(Size%3a%20512).html>) 284.95 ns 232.89 ns 0.82 0.03 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).List(Size%3a%20512).html>) 70.34 ns 58.72 ns 0.83 0.03 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).SortedList(Size%3a%20512).html>) 141.64 ns 116.20 ns 0.82 0.03 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(Int32).ConcurrentDictionary(Size%3a%20512).html>) 172.42 ns 148.52 ns 0.86 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorGivenSize<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 49.660601695413526 < 58.54124767954895. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/3/2022 11:42:39 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 23.948266092692386 (T) = (0 -50.48097669706972) / Math.Sqrt((2.434270134966209 / (299)) + (1.8094415440906293 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19676429645599164 = (62.84702793257238 - 50.48097669706972) / 62.84702793257238 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 58.359634938297965 < 66.62605187295436. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/22/2022 2:38:09 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.74370850755075 (T) = (0 -58.75084490255099) / Math.Sqrt((2.395409457191092 / (299)) + (1.4700129932993564 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1806579555070111 = (71.70490675711164 - 58.75084490255099) / 71.70490675711164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 58.05739309560086 < 66.39881940516923. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/22/2022 2:38:09 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.87821820425477 (T) = (0 -59.01804694521447) / Math.Sqrt((2.7559145585207974 / (299)) + (2.1742555969024067 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17980804064219771 = (71.95638322451332 - 59.01804694521447) / 71.95638322451332 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 232.89017354490215 < 267.14306396789414. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/22/2022 2:38:09 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.59914764146587 (T) = (0 -238.06287160798203) / Math.Sqrt((49.56487209321379 / (299)) + (34.937024815619424 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17734102400006518 = (289.38220885345436 - 238.06287160798203) / 289.38220885345436 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 58.717530675572874 < 66.4633661806415. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 25.058067195052402 (T) = (0 -59.099963292741855) / Math.Sqrt((3.642817718415208 / (299)) + (1.686048544776118 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17579167608027982 = (71.70512791193107 - 59.099963292741855) / 71.70512791193107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 116.1994343830523 < 132.98971189901184. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 33.24917796810442 (T) = (0 -117.61690818296245) / Math.Sqrt((9.242613904648247 / (299)) + (4.075785553278215 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.18124104027390078 = (143.6526669855424 - 117.61690818296245) / 143.6526669855424 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 148.51942424832717 < 164.60082928235195. IsChangePoint: Marked as a change because one of 3/11/2022 8:34:45 AM, 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 16.8054130334634 (T) = (0 -151.66832472071104) / Math.Sqrt((13.10086912481841 / (299)) + (15.626706052068819 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14322716851613615 = (177.02279898165457 - 151.66832472071104) / 177.02279898165457 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayCreate1D - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Array.ArrayCreate1D.html>) 462.04 ns 387.06 ns 0.84 0.01 True
[ArrayCreate3D - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Array.ArrayCreate3D.html>) 553.87 ns 452.05 ns 0.82 0.01 False
[ArrayCreate2D - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Array.ArrayCreate2D.html>) 526.51 ns 427.85 ns 0.81 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Array.ArrayCreate1D ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 387.0566580769651 < 443.51021086013907. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 78.3099867927844 (T) = (0 -386.14188544517793) / Math.Sqrt((34.17260364186813 / (299)) + (7.300828219938963 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17909368190630984 = (470.3848379945195 - 386.14188544517793) / 470.3848379945195 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Tests.Perf_Array.ArrayCreate3D ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 452.0486912830826 < 528.307350249956. IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 63.580456406284114 (T) = (0 -454.9566359736777) / Math.Sqrt((563.5606883089117 / (299)) + (17.998934465073948 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.22780280327919072 = (589.1715715955505 - 454.9566359736777) / 589.1715715955505 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Array.ArrayCreate2D ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 427.8502870913745 < 504.43598272124524. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 78.1524828821813 (T) = (0 -429.2581740881323) / Math.Sqrt((814.6401075000717 / (299)) + (1.9207686075469708 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.23971064513498028 = (564.598427350521 - 429.2581740881323) / 564.598427350521 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) 2.66 μs 2.41 μs 0.91 0.02 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 2.69 μs 2.49 μs 0.93 0.01 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 2.59 μs 2.40 μs 0.93 0.00 False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 2.83 μs 2.65 μs 0.94 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.411314619520265 < 2.5457292508360405. IsChangePoint: Marked as a change because one of 3/15/2022 4:36:24 AM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 18.894857133398034 (T) = (0 -2430.8796390888024) / Math.Sqrt((6722.121961255963 / (299)) + (1186.8031144388065 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.09723732334102447 = (2692.711719192045 - 2430.8796390888024) / 2692.711719192045 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.4934232362408313 < 2.548612071992626. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 32.24321136921371 (T) = (0 -2493.1223251572524) / Math.Sqrt((6925.801416748555 / (299)) + (209.44861584113326 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08611338570125773 = (2728.0433766614624 - 2493.1223251572524) / 2728.0433766614624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.3991039471508224 < 2.4611448409620453. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 54.02704128990273 (T) = (0 -2390.6834626502646) / Math.Sqrt((3284.6643768155895 / (299)) + (99.353289305552 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.10184888196666442 = (2661.7830949040053 - 2390.6834626502646) / 2661.7830949040053 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.652885751954901 < 2.7105921908549004. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 32.03891445601007 (T) = (0 -2637.267711400113) / Math.Sqrt((4093.781134877954 / (299)) + (272.20141800290617 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08095784560061661 = (2869.582966108483 - 2637.267711400113) / 2869.582966108483 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.CtorGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).Queue(Size%3a%20512).html>) 139.66 ns 113.86 ns 0.82 0.01 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).List(Size%3a%20512).html>) 139.04 ns 113.26 ns 0.81 0.01 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).SortedList(Size%3a%20512).html>) 284.44 ns 234.97 ns 0.83 0.00 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).Stack(Size%3a%20512).html>) 139.02 ns 113.42 ns 0.82 0.00 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).ConcurrentDictionary(Size%3a%20512).html>) 184.01 ns 153.79 ns 0.84 0.02 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).Array(Size%3a%20512).html>) 124.57 ns 100.61 ns 0.81 0.01 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSize(String).Dictionary(Size%3a%20512).html>) 485.24 ns 396.88 ns 0.82 0.00 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorGivenSize<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 113.85676532712627 < 131.52451705925358. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/22/2022 2:38:09 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 32.75913042205114 (T) = (0 -115.11995810093983) / Math.Sqrt((6.494577993502345 / (299)) + (3.8691584085165065 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1774154369010007 = (139.94908641032322 - 115.11995810093983) / 139.94908641032322 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 113.25636696663001 < 131.74795153087067. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 34.587023067788344 (T) = (0 -114.29104160187718) / Math.Sqrt((4.748182176263031 / (299)) + (3.536417159229804 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17928592111594824 = (139.25804923108174 - 114.29104160187718) / 139.25804923108174 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 234.9728002768228 < 271.359635327986. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 33.614212848465854 (T) = (0 -238.0354778320306) / Math.Sqrt((25.65183010705056 / (299)) + (15.830213733755693 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1778696056230486 = (289.5349441646966 - 238.0354778320306) / 289.5349441646966 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 113.42256897371999 < 131.8147715735227. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.399090801774303 (T) = (0 -114.97656505075797) / Math.Sqrt((5.457190683664726 / (299)) + (5.617831923568526 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1775614426809129 = (139.79958019666353 - 114.97656505075797) / 139.79958019666353 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 153.78876259697742 < 169.43833035115372. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/15/2022 9:38:29 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 40.75368658299124 (T) = (0 -154.51120350866788) / Math.Sqrt((24.539806528253138 / (299)) + (3.1338669634481042 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16105784721044014 = (184.17384678419592 - 154.51120350866788) / 184.17384678419592 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 100.6104235332815 < 118.22113985777445. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.466541971383275 (T) = (0 -101.7941987158737) / Math.Sqrt((6.287445186904657 / (299)) + (7.656550533729316 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.18899112676131405 = (125.51551786279275 - 101.7941987158737) / 125.51551786279275 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorGivenSize<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 396.8806983661303 < 465.95701385329727. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 67.25902633856866 (T) = (0 -400.84126529197465) / Math.Sqrt((62.406415604034066 / (299)) + (12.845982648964062 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19347371923407142 = (496.9971529152284 - 400.84126529197465) / 496.9971529152284 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetChars - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_Encoding.GetChars(size%3a%20512%2c%20encName%3a%20%22ascii%22).html>) 78.77 ns 72.56 ns 0.92 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 72.56491561398103 < 74.49978336516452. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 43.493956667249975 (T) = (0 -72.7497093430653) / Math.Sqrt((6.3746025143525635 / (299)) + (0.04890991846486553 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.09139428891329686 = (80.06741368162412 - 72.7497093430653) / 80.06741368162412 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.ReadOnlyMemory<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.ReadOnlyMemory(Char).ToArray(Size%3a%20512).html>) 56.91 ns 51.46 ns 0.90 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlyMemory&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlyMemory<Char>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 51.457133954272415 < 54.509177926405414. IsChangePoint: Marked as a change because one of 3/16/2022 5:02:20 PM, 3/20/2022 10:41:02 AM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/23/2022 12:40:50 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 10.243189325696767 (T) = (0 -52.12682460551613) / Math.Sqrt((3.503455066859386 / (299)) + (0.7153867122203477 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.062197969804872265 = (55.584038983867565 - 52.12682460551613) / 55.584038983867565 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.CtorDefaultSizeNonGeneric

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorDefaultSizeNonGeneric.Queue.html>) 18.17 ns 15.70 ns 0.86 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorDefaultSizeNonGeneric.Queue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.697437613477783 < 17.139067401536188. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/8/2022 2:40:20 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 38.808014530859936 (T) = (0 -15.664409584958674) / Math.Sqrt((0.2096081867157061 / (299)) + (0.02446674045703354 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13829272929282913 = (18.178342132477866 - 15.664409584958674) / 18.178342132477866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Tests.Perf_GC<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AllocateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_GC(Char).AllocateArray(length%3a%201000%2c%20pinned%3a%20False).html>) 82.80 ns 69.90 ns 0.84 0.02 False
[AllocateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_GC(Char).AllocateArray(length%3a%2010000%2c%20pinned%3a%20True).html>) 900.61 ns 807.08 ns 0.90 0.03 False
[AllocateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_GC(Char).AllocateArray(length%3a%2010000%2c%20pinned%3a%20False).html>) 529.60 ns 428.18 ns 0.81 0.01 True
[AllocateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_GC(Char).AllocateArray(length%3a%201000%2c%20pinned%3a%20True).html>) 213.33 ns 193.19 ns 0.91 0.02 False
[NewOperator_Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_GC(Char).NewOperator_Array(length%3a%2010000).html>) 526.49 ns 422.78 ns 0.80 0.00 True
[AllocateUninitializedArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_GC(Char).AllocateUninitializedArray(length%3a%201000%2c%20pinned%3a%20False).html>) 61.60 ns 49.64 ns 0.81 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 69.9035487038772 < 79.99439809909599. IsChangePoint: Marked as a change because one of 4/16/2022 2:55:52 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 13.390279209436247 (T) = (0 -71.31294515642111) / Math.Sqrt((6.18833880660269 / (299)) + (8.650126943319572 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1738746340268495 = (86.32218316213621 - 71.31294515642111) / 86.32218316213621 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 807.0770759541924 < 851.4106872816451. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/31/2022 1:23:38 AM, 4/8/2022 10:19:59 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 40.210973754193716 (T) = (0 -810.6643223303935) / Math.Sqrt((2087.9178051295626 / (299)) + (26.05448426868153 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13963134011190642 = (942.2290235860462 - 810.6643223303935) / 942.2290235860462 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 428.18226694372777 < 509.9241263725888. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 87.33821898849705 (T) = (0 -430.96691672368354) / Math.Sqrt((27.417083483271707 / (299)) + (9.903196039781317 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19918927206070747 = (538.1632659101366 - 430.96691672368354) / 538.1632659101366 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 193.18655261011594 < 202.72445098572854. IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 16.844321940101707 (T) = (0 -192.56372605542242) / Math.Sqrt((27.805342037001203 / (299)) + (14.320892193609202 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11341909752382942 = (217.19814347185098 - 192.56372605542242) / 217.19814347185098 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_GC<Char>.NewOperator_Array(length: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 422.7788058102548 < 502.7982837310777. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 78.97513392075562 (T) = (0 -426.7447845131142) / Math.Sqrt((29.522504420916285 / (299)) + (11.749940786234392 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19789462556310128 = (532.0308255167863 - 426.7447845131142) / 532.0308255167863 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 1000, pinned: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 49.63572481085699 < 58.718194207845464. IsChangePoint: Marked as a change because one of 4/18/2022 8:59:00 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.08693657190154 (T) = (0 -50.04305397803235) / Math.Sqrt((1.4402515529922113 / (299)) + (1.395445575973292 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19651141618526677 = (62.28222153504943 - 50.04305397803235) / 62.28222153504943 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.IO.Compression.Brotli

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Decompress_WithState - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Compression.Brotli.Decompress_WithState(level%3a%20Fastest%2c%20file%3a%20%22TestDocument.pdf%22).html>) 330.53 μs 298.55 μs 0.90 0.00 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Compression.Brotli.Decompress_WithState(level: Fastest, file: "TestDocument.pdf") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 298.54659703504046 < 313.48068339223704. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 273.2993615109449 (T) = (0 -298188.64603388525) / Math.Sqrt((292327.4903247645 / (299)) + (88894.37181767204 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.09680998608226249 = (330150.5125598568 - 298188.64603388525) / 330150.5125598568 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Clear - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Int32).Clear(Size%3a%20512).html>) 43.41 ns 26.93 ns 0.62 0.01 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.Clear(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 26.926245222145404 < 41.264085958836645. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 86.31970962510029 (T) = (0 -26.504594510656926) / Math.Sqrt((0.10702088307765226 / (299)) + (0.267329457992935 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.39002964951066676 = (43.45226696575381 - 26.504594510656926) / 43.45226696575381 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.CtorGivenSizeNonGeneric

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorGivenSizeNonGeneric.Queue(Size%3a%20512).html>) 138.26 ns 111.83 ns 0.81 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorGivenSizeNonGeneric*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorGivenSizeNonGeneric.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 111.8280402233165 < 131.0375307368853. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/22/2022 2:38:09 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.516396496214863 (T) = (0 -113.08111238392641) / Math.Sqrt((6.538732546954415 / (299)) + (5.8326617602600805 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.18368346856559867 = (138.52605947502306 - 113.08111238392641) / 138.52605947502306 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsDefinedClassMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Reflection.Attributes.IsDefinedClassMiss.html>) 117.09 ns 102.70 ns 0.88 0.04 False
[IsDefinedMethodOverrideHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Reflection.Attributes.IsDefinedMethodOverrideHit.html>) 576.52 ns 531.81 ns 0.92 0.01 False
[IsDefinedClassMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Reflection.Attributes.IsDefinedClassMissInherit.html>) 939.79 ns 840.01 ns 0.89 0.02 False
[IsDefinedMethodOverrideHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit.html>) 577.47 ns 529.47 ns 0.92 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Attributes*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Attributes.IsDefinedClassMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 102.70150509105083 < 112.09917650857574. IsChangePoint: Marked as a change because one of 3/20/2022 10:41:02 AM, 4/18/2022 2:31:51 PM, 5/3/2022 11:42:39 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 14.567294556827644 (T) = (0 -101.28719234428368) / Math.Sqrt((34.95982949781632 / (299)) + (8.584997628025096 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14287697317412293 = (118.17112500100873 - 101.28719234428368) / 118.17112500100873 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodOverrideHit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 531.8073616155215 < 544.1358598926726. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 42.56371415699643 (T) = (0 -527.8592101049477) / Math.Sqrt((913.4848953477239 / (299)) + (8.801574088662795 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14343251438308277 = (616.2494128816631 - 527.8592101049477) / 616.2494128816631 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedClassMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 840.0123749058431 < 912.3325949273064. IsChangePoint: Marked as a change because one of 3/9/2022 2:24:14 PM, 4/27/2022 8:46:50 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 17.93979483677456 (T) = (0 -835.9389073477943) / Math.Sqrt((436.42376448769215 / (299)) + (209.42358899288448 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1073124182340874 = (936.4294120616552 - 835.9389073477943) / 936.4294120616552 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 529.465972259701 < 547.982563046952. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 31.57311704771213 (T) = (0 -529.8272899818885) / Math.Sqrt((917.6285576507915 / (299)) + (30.465805040615354 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13966546017193843 = (615.8386830404075 - 529.8272899818885) / 615.8386830404075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CreateAddAndClear(Int32).ImmutableArray(Size%3a%20512).html>) 30.60 μs 26.81 μs 0.88 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 26.806999143835615 < 28.876034670326558. IsChangePoint: Marked as a change because one of 3/23/2022 10:40:09 PM, 3/30/2022 6:37:35 PM, 4/16/2022 2:55:52 PM, 4/25/2022 11:05:49 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 39.85309015642178 (T) = (0 -26937.558182940218) / Math.Sqrt((467744.50141303043 / (299)) + (57390.390886577516 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1275385431263616 = (30875.356121137713 - 26937.558182940218) / 30875.356121137713 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorFromCollection(String).ImmutableArray(Size%3a%20512).html>) 285.03 ns 250.34 ns 0.88 0.03 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorFromCollection(String).Stack(Size%3a%20512).html>) 272.01 ns 243.70 ns 0.90 0.01 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorFromCollection(String).Queue(Size%3a%20512).html>) 272.87 ns 245.04 ns 0.90 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 250.3362033874461 < 266.59854184328435. IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 40.31152184755707 (T) = (0 -251.27155934772924) / Math.Sqrt((50.09127017252582 / (299)) + (3.024021785862235 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11049454093868913 = (282.48456126721743 - 251.27155934772924) / 282.48456126721743 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 243.70261139331708 < 256.1126271191424. IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 81.95243097222377 (T) = (0 -243.38763389859582) / Math.Sqrt((30.842757922920363 / (299)) + (0.36641145715826745 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11721424548405182 = (275.70407956124177 - 243.38763389859582) / 275.70407956124177 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 245.04309901605475 < 251.59769854136377. IsChangePoint: Marked as a change because one of 4/6/2022 2:58:43 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 24.15617080570958 (T) = (0 -244.82661960238212) / Math.Sqrt((39.045004416833365 / (299)) + (10.032206986866187 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1098313867410475 = (275.03398340013297 - 244.82661960238212) / 275.03398340013297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.Memory<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Memory(Char).ToArray(Size%3a%20512).html>) 56.42 ns 50.57 ns 0.90 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Memory&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Memory<Char>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 50.56593594092142 < 53.58424790528686. IsChangePoint: Marked as a change because one of 3/10/2022 7:26:59 AM, 3/17/2022 6:54:53 PM, 3/20/2022 10:41:02 AM, 4/8/2022 7:02:49 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 8.248728886863336 (T) = (0 -51.083157472962895) / Math.Sqrt((3.667922905075342 / (299)) + (1.2060504008573176 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.06487089844625579 = (54.62685033337829 - 51.083157472962895) / 54.62685033337829 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ctor_capacity - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_StringBuilder.ctor_capacity(length%3a%20100).html>) 16.82 ns 14.35 ns 0.85 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.351750049981488 < 14.929480811512844. IsChangePoint: Marked as a change because one of 3/22/2022 11:55:12 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 19.953299310613986 (T) = (0 -14.4055468586046) / Math.Sqrt((0.3957564518941707 / (299)) + (0.05419331287612359 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11651458235500935 = (16.305358946391976 - 14.4055468586046) / 16.305358946391976 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteByteArrayAsBase64_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes%3a%201000).html>) 186.69 ns 173.06 ns 0.93 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 173.0645049832678 < 178.3682461208051. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 24.63347832620463 (T) = (0 -174.1577847050639) / Math.Sqrt((21.886635977018106 / (299)) + (2.277277166828521 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08197215580604525 = (189.70860830259198 - 174.1577847050639) / 189.70860830259198 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayIntArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size%3a%20512).html>) 104.76 ns 93.98 ns 0.90 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 93.97710763401658 < 101.33349462095859. IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/7/2022 1:02:48 PM, 3/11/2022 8:42:43 PM, 4/8/2022 7:02:49 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 14.717811457210578 (T) = (0 -94.84565957953596) / Math.Sqrt((4.315586471082173 / (299)) + (6.210863424612467 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1284280679443787 = (108.82137904078716 - 94.84565957953596) / 108.82137904078716 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)