search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 28 Improvements on 1/11/2023 1:25:06 PM #11900

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).SquareRootBenchmark.html>) 52.18 ns 49.14 ns 0.94 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 49.13725299437098 < 49.51693368002621. IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 33.53632538261448 (T) = (0 -48.87064411802077) / Math.Sqrt((1.922153256856369 / (206)) + (0.15513209080999818 / (30))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (30) - 2, .975) and 0.07633025998859219 = (52.909218523730345 - 48.87064411802077) / 52.909218523730345 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetMethod20 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod20.html>) 1.70 ms 1.60 ms 0.94 0.00 False
[GetMethod10 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod10.html>) 843.00 μs 796.21 μs 0.94 0.00 False
[GetMethod3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod3.html>) 252.43 μs 238.11 μs 0.94 0.00 False
[GetMethod2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod2.html>) 167.99 μs 158.76 μs 0.95 0.00 False
[GetMethod4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod4.html>) 341.13 μs 317.66 μs 0.93 0.00 False
[GetMethod12 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod12.html>) 1.01 ms 958.17 μs 0.95 0.00 False
[GetMethod1 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod1.html>) 85.10 μs 79.81 μs 0.94 0.00 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.GetMember.GetMethod20 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.5996285833333332 < 1.6113399666666666. IsChangePoint: Marked as a change because one of 11/2/2022 5:57:48 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 54.4492315842968 (T) = (0 -1584690.2802770147) / Math.Sqrt((416610400.259604 / (299)) + (102369541.01163894 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07004632738720386 = (1704052.92967409 - 1584690.2802770147) / 1704052.92967409 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod10 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 796.2149759615385 < 802.8606555059524. IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 86.70710463534287 (T) = (0 -789184.2755971462) / Math.Sqrt((75173660.9447533 / (299)) + (7563712.981649724 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07232514797319115 = (850712.1583310309 - 789184.2755971462) / 850712.1583310309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 238.11185606060607 < 239.75054553091397. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 73.85965513638538 (T) = (0 -236540.57107689502) / Math.Sqrt((5861239.662705289 / (299)) + (1155671.4731511152 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07001030446363521 = (254347.51827058897 - 236540.57107689502) / 254347.51827058897 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 158.76027687590187 < 160.32022772435695. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 106.61283921679878 (T) = (0 -157691.05911971128) / Math.Sqrt((2518320.8317912524 / (299)) + (125644.01355194907 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07056502854358107 = (169663.35888202093 - 157691.05911971128) / 169663.35888202093 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 317.6594666666667 < 321.28366666666665. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 98.15063898479023 (T) = (0 -315040.17150030535) / Math.Sqrt((9223179.589288805 / (299)) + (867825.7863431786 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07077884388048666 = (339036.8045599964 - 315040.17150030535) / 339036.8045599964 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod12 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 958.1671568627453 < 0.9635649637920672. IsChangePoint: Marked as a change because one of 11/2/2022 5:57:48 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 77.57565820396022 (T) = (0 -948380.545033398) / Math.Sqrt((117505400.27520803 / (299)) + (13818014.36991114 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07026333431613657 = (1020052.8601674767 - 948380.545033398) / 1020052.8601674767 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod1 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 79.80739795918369 < 80.46444221240472. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 91.37853144344038 (T) = (0 -79137.51598469002) / Math.Sqrt((551546.5297457653 / (299)) + (71687.60164591286 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06988520214869896 = (85083.60061307385 - 79137.51598469002) / 85083.60061307385 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).SortedSet(Size%3a%20512).html>) 11.13 μs 10.31 μs 0.93 0.01 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).ImmutableArray(Size%3a%20512).html>) 416.55 ns 339.03 ns 0.81 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.310847860445369 < 10.434198161921689. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 30.013868997003083 (T) = (0 -10390.347954235114) / Math.Sqrt((11680.686790975396 / (299)) + (11023.334813610489 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.055035531649387105 = (10995.490626616824 - 10390.347954235114) / 10995.490626616824 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 339.0250462516088 < 395.8927582397259. IsChangePoint: Marked as a change because one of 11/6/2022 12:38:35 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 47.39726068469033 (T) = (0 -338.97527417876375) / Math.Sqrt((1323.3209657388954 / (299)) + (0.17751105562223263 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.2274148005788151 = (438.75455345600915 - 338.97527417876375) / 438.75455345600915 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForEach_Array - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array.html>) 5.58 ns 2.81 ns 0.50 0.21 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.812298401926418 < 5.293653042160978. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 40.6280166043294 (T) = (0 -2.836287956548215) / Math.Sqrt((0.32691706115929636 / (299)) + (0.09895184833086902 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.48699030930050063 = (5.528721987065931 - 2.836287956548215) / 5.528721987065931 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22(%3f%3a(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%5c%5c.)%7b3%7d(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%22%2c%20Options%3a%20NonBacktracking).html>) 19.81 ms 18.39 ms 0.93 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.392118333333332 < 18.886030875. IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/9/2022 4:06:23 PM, 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 62.71710386650591 (T) = (0 -18399251.5721917) / Math.Sqrt((423581238963.879 / (299)) + (3879831199.5120273 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11818594332827699 = (20865228.256438732 - 18399251.5721917) / 20865228.256438732 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConvertTryFromBase64Chars - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Text.Tests.Base64Tests.ConvertTryFromBase64Chars(NumberOfBytes%3a%201000).html>) 1.02 μs 922.45 ns 0.91 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Base64Tests.ConvertTryFromBase64Chars(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 922.4493964500055 < 0.9658044521711197. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 304.67587727625437 (T) = (0 -922.2858756549834) / Math.Sqrt((27.756495217655534 / (299)) + (0.1405546932438531 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09351311436674375 = (1017.4288125643321 - 922.2858756549834) / 1017.4288125643321 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).SquareRootBenchmark.html>) 48.84 ns 44.65 ns 0.91 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 44.648498612351574 < 46.358376876152406. IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 41.72680022078616 (T) = (0 -44.47673423296093) / Math.Sqrt((1.9367296251562889 / (206)) + (0.15524530048180696 / (30))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (30) - 2, .975) and 0.10174385206475582 = (49.51453361627009 - 44.47673423296093) / 49.51453361627009 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForeachOverList100Elements - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.LowLevelPerf.ForeachOverList100Elements.html>) 13.68 ms 9.65 ms 0.71 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ForeachOverList100Elements ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.648181818181818 < 12.98812347027972. IsChangePoint: Marked as a change because one of 11/4/2022 9:11:54 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 44.721936217751434 (T) = (0 -9649497.042055167) / Math.Sqrt((1687617042460.202 / (299)) + (202941842.21563405 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.25837996379939515 = (13011375.867742902 - 9649497.042055167) / 13011375.867742902 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).SortedSet(Size%3a%20512).html>) 10.61 μs 9.80 μs 0.92 0.00 False
[List - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).List(Size%3a%20512).html>) 657.25 ns 494.28 ns 0.75 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.804413896405102 < 10.093294170350896. IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 78.01083521844235 (T) = (0 -9855.945146457338) / Math.Sqrt((13460.959138111797 / (299)) + (1666.4198334151833 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07353811783257583 = (10638.262983253784 - 9855.945146457338) / 10638.262983253784 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 494.2825510835252 < 624.3906213247813. IsChangePoint: Marked as a change because one of 10/6/2022 9:08:45 AM, 11/4/2022 9:11:54 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 67.5893815885865 (T) = (0 -494.1680230637391) / Math.Sqrt((1340.4452909697102 / (299)) + (0.03826851594526151 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.2245882142102639 = (637.2975393460679 - 494.1680230637391) / 637.2975393460679 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseDecimal - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value%3a%20123456.789).html>) 50.10 ns 47.20 ns 0.94 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 47.203624511793755 < 47.57585682387295. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 43.50944375068865 (T) = (0 -46.583792165872445) / Math.Sqrt((0.1855914325265376 / (299)) + (0.15510705779799583 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06635948449664585 = (49.894784333301665 - 46.583792165872445) / 49.894784333301665 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 98.54 ns 90.96 ns 0.92 0.02 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 101.22 ns 90.66 ns 0.90 0.03 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.95680827724503 < 93.72393303987727. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 30.986991096223715 (T) = (0 -91.47970803364761) / Math.Sqrt((2.3472984930626146 / (299)) + (1.1514230602567184 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0678876520733316 = (98.1423625994541 - 91.47970803364761) / 98.1423625994541 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.65758727352197 < 94.41956393533036. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 26.481166051322692 (T) = (0 -91.73357854715752) / Math.Sqrt((4.035066308615414 / (299)) + (1.4790468400496002 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06745978877521423 = (98.36956888612436 - 91.73357854715752) / 98.36956888612436 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsyncThenReadAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync.html>) 81.03 ns 71.38 ns 0.88 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 71.3814782152132 < 77.0222556970939. IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 26.54738389663159 (T) = (0 -72.49296554615626) / Math.Sqrt((2.1297573680244253 / (299)) + (2.222732664198492 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09449961036367678 = (80.0584586995834 - 72.49296554615626) / 80.0584586995834 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Depth

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth%3a%201).html>) 116.55 ns 95.21 ns 0.82 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 1) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 95.21462909950091 < 97.59717974275476. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 33.48006836787368 (T) = (0 -94.94584073707055) / Math.Sqrt((17.109675590790197 / (299)) + (0.6967192566047952 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09092150634876646 = (104.44185117143071 - 94.94584073707055) / 104.44185117143071 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Base64DecodeInPlace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes%3a%20200000000).html>) 185.55 ms 167.93 ms 0.91 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes: 200000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 167.92996923076922 < 176.3657015595238. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 571.3926542051051 (T) = (0 -168208916.82905987) / Math.Sqrt((29737041076.463184 / (299)) + (24820504435.870773 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09372194994145878 = (185604094.48093152 - 168208916.82905987) / 185604094.48093152 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IndexerSet(Int32).ConcurrentDictionary(Size%3a%20512).html>) 26.25 μs 18.95 μs 0.72 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.945059813463097 < 25.06043489865489. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 377.75873870864575 (T) = (0 -18947.894978337652) / Math.Sqrt((78854.55476090226 / (299)) + (3887.428992815522 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.28335193862400165 = (26439.609620874147 - 18947.894978337652) / 26439.609620874147 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 332.79 ns 210.30 ns 0.63 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 210.30153764000326 < 316.1422540727137. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 631.6561856143796 (T) = (0 -209.75297510725596) / Math.Sqrt((0.21827446183759924 / (299)) + (1.1183999513564562 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.3699251074999984 = (332.9016559841026 - 209.75297510725596) / 332.9016559841026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Replace_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Replace_Char(text%3a%20%22yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi79.html>) 253.60 ns 218.22 ns 0.86 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 218.22474639396805 < 241.41514821472683. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 26.9031268242633 (T) = (0 -221.71395882087134) / Math.Sqrt((28.40402280838542 / (299)) + (52.2997567516413 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.14127759796934078 = (258.19049124207595 - 221.71395882087134) / 258.19049124207595 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Copy_PassAsArgumentAndReturn_FromValueTaskSource - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource.html>) 12.18 ns 11.05 ns 0.91 0.13 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.046255535442077 < 11.315047026047967. IsChangePoint: Marked as a change because one of 10/7/2022 7:50:02 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 16.245918013459903 (T) = (0 -10.263171557699913) / Math.Sqrt((0.3941995262939758 / (299)) + (0.27037271049393113 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.13859220241049774 = (11.914416826060302 - 10.263171557699913) / 11.914416826060302 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableHashSet(Size%3a%20512).html>) 269.98 μs 238.87 μs 0.88 0.19 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 238.87289772727274 < 249.3374735863095. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 19.00728408864786 (T) = (0 -244210.8531595064) / Math.Sqrt((114496930.44370933 / (299)) + (56161918.478256464 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1046460470893353 = (272753.4204384899 - 244210.8531595064) / 272753.4204384899 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)