search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 30 Improvements on 1/11/2023 1:25:06 PM #11909

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.IterateForEach(String).ImmutableArray(Size%3a%20512).html>) 436.71 ns 354.25 ns 0.81 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 354.25085637962246 < 414.78702714032244. IsChangePoint: Marked as a change because one of 11/6/2022 12:38:35 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 47.64623605786968 (T) = (0 -354.23771948864) / Math.Sqrt((1460.425876211639 / (299)) + (0.33449871691804056 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.22935349491350301 = (459.66304544374793 - 354.23771948864) / 459.66304544374793 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 104.63 ns 97.47 ns 0.93 0.05 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 106.37 ns 99.39 ns 0.93 0.07 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 104.98 ns 97.58 ns 0.93 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 97.46756583238589 < 99.2008504875939. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 23.16849379368833 (T) = (0 -97.40342364740849) / Math.Sqrt((3.9086345545298378 / (254)) + (2.134314150352824 / (29))) is greater than 1.968442091833948 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (254) + (29) - 2, .975) and 0.06625393189648365 = (104.31468144785828 - 97.40342364740849) / 104.31468144785828 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 99.38537008911801 < 101.2142298318966. IsChangePoint: Marked as a change because one of 11/14/2022 7:25:10 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 21.70764024064691 (T) = (0 -97.69478060585476) / Math.Sqrt((4.354751077349012 / (299)) + (2.405942573100758 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06488869585437064 = (104.47395959469685 - 97.69478060585476) / 104.47395959469685 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 97.58042831925681 < 99.51120820341535. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 25.511211134760373 (T) = (0 -97.18869610745058) / Math.Sqrt((3.13005933698212 / (259)) + (1.9184341410000223 / (29))) is greater than 1.9682932552436228 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (259) + (29) - 2, .975) and 0.06839967296594923 = (104.32445469064145 - 97.18869610745058) / 104.32445469064145 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseDecimal - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value%3a%20123456.789).html>) 52.03 ns 49.31 ns 0.95 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 49.310924422815354 < 49.38420873356681. IsChangePoint: Marked as a change because one of 11/30/2022 8:56:07 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 39.99380351066706 (T) = (0 -48.953751748545926) / Math.Sqrt((0.25059198914495845 / (299)) + (0.19056976940764878 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06570301447728026 = (52.39634988349804 - 48.953751748545926) / 52.39634988349804 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForEach_Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array.html>) 5.62 ns 2.92 ns 0.52 0.31 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.9150643297492445 < 5.548712850580092. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 12:41:43 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 35.10370290667445 (T) = (0 -3.0633682662945874) / Math.Sqrt((0.2753074590133619 / (299)) + (0.15992682755326962 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.4789691810489396 = (5.87943775084507 - 3.0633682662945874) / 5.87943775084507 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Copy_PassAsArgumentAndReturn_FromValueTaskSource - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource.html>) 12.44 ns 10.44 ns 0.84 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.441778938906241 < 11.919045715825131. IsChangePoint: Marked as a change because one of 10/7/2022 4:12:34 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 33.904055157305194 (T) = (0 -10.498078634827792) / Math.Sqrt((0.8814738212167278 / (259)) + (0.05360346899737624 / (29))) is greater than 1.9682932552436228 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (259) + (29) - 2, .975) and 0.1896551078898937 = (12.95507473057701 - 10.498078634827792) / 12.95507473057701 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetMethod15 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod15.html>) 1.34 ms 1.26 ms 0.94 0.02 False
[GetMethod1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod1.html>) 88.95 μs 83.89 μs 0.94 0.03 False
[GetMethod4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod4.html>) 354.69 μs 334.31 μs 0.94 0.03 False
[GetMethod2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod2.html>) 178.88 μs 166.98 μs 0.93 0.02 False
[GetMethod5 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod5.html>) 443.34 μs 418.20 μs 0.94 0.03 False
[GetMethod3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod3.html>) 266.03 μs 252.03 μs 0.95 0.04 False
[GetMethod20 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod20.html>) 1.79 ms 1.68 ms 0.94 0.02 False
[GetMethod10 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.GetMember.GetMethod10.html>) 894.29 μs 838.43 μs 0.94 0.02 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.GetMember.GetMethod15 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.2613100906593406 < 1.2714535111514136. IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 59.803124891096665 (T) = (0 -1248528.233229674) / Math.Sqrt((466685965.94924617 / (299)) + (33678590.166900195 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07324011673618035 = (1347197.1065824144 - 1248528.233229674) / 1347197.1065824144 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod1 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 83.89336131907307 < 84.32073938328598. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 58.91352544767675 (T) = (0 -83283.01056677512) / Math.Sqrt((2117524.1684576115 / (299)) + (108767.74052983349 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06857612845863383 = (89414.72632535634 - 83283.01056677512) / 89414.72632535634 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 334.3149825081832 < 337.2648212144887. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 51.01948578525812 (T) = (0 -331819.602045543) / Math.Sqrt((26804761.666409783 / (299)) + (4548987.058936672 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07092536954109034 = (357150.6434113298 - 331819.602045543) / 357150.6434113298 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 166.9807903368794 < 169.45545959280304. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 63.76011894637304 (T) = (0 -166142.21550850075) / Math.Sqrt((5971706.325738873 / (299)) + (466350.08974261483 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06842155145445417 = (178344.84660727732 - 166142.21550850075) / 178344.84660727732 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod5 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 418.2035471491228 < 421.4453805584491. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 51.79676128878649 (T) = (0 -414820.94543736667) / Math.Sqrt((60566843.69941503 / (299)) + (4306234.781853493 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06888625681258552 = (445510.495868464 - 414820.94543736667) / 445510.495868464 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 252.0296644179894 < 253.98815504224672. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 44.41607149451991 (T) = (0 -249149.5717236881) / Math.Sqrt((16591204.79087715 / (299)) + (3852446.468087569 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07180907471974357 = (268424.91661773174 - 249149.5717236881) / 268424.91661773174 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod20 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.6816703862500004 < 1.695174769832176. IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 8.295084076094899 (T) = (0 -1683295.6757569944) / Math.Sqrt((766917923.5533457 / (299)) + (5337193198.814413 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06307103442432657 = (1796609.7085308211 - 1683295.6757569944) / 1796609.7085308211 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod10 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 838.4305699561404 < 845.1870572387248. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 63.59428250729882 (T) = (0 -830741.0028912299) / Math.Sqrt((140295281.19077963 / (299)) + (14798690.075860132 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07042733224803666 = (893680.5391452145 - 830741.0028912299) / 893680.5391452145 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromReader(Mode%3a%20Reflection).html>) 36.51 μs 32.84 μs 0.90 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 32.840823074903035 < 34.29978542360296. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 18.472241969837338 (T) = (0 -33641.44984171547) / Math.Sqrt((551466.5703261832 / (120)) + (251640.12863651547 / (29))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (120) + (29) - 2, .975) and 0.05949587984994302 = (35769.59326488436 - 33641.44984171547) / 35769.59326488436 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.IndexerSet(Int32).ConcurrentDictionary(Size%3a%20512).html>) 29.89 μs 23.39 μs 0.78 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 23.393536816939886 < 28.36917053162923. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 101.3107772816823 (T) = (0 -23537.264841395092) / Math.Sqrt((18163.019342783784 / (299)) + (111229.30243757491 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.21177421847315053 = (29861.06949686641 - 23537.264841395092) / 29861.06949686641 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22(%3f%3a(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%5c%5c.)%7b3%7d(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%22%2c%20Options%3a%20NonBacktracking).html>) 20.75 ms 19.15 ms 0.92 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern%3a%20%22%5b%5c%5cw%5d%2b%3a%2f%2f%5b%5e%2f%5c%5cs%3f%23%5d%2b%5b%5e%5c%5cs%3f%23%5d%2b(%3f%3a%5c%5c%3f%5b%5e%5c%5cs%23%5d)%3f(%3f%3a%23%5b%5e%5c%5cs%5d)%3f%22%2c%20Options%3a%20Compiled).html>) 2.40 ms 2.25 ms 0.94 0.00 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.147947016666667 < 19.703552147916668. IsChangePoint: Marked as a change because one of 11/9/2022 4:06:23 PM, 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 47.79405070703091 (T) = (0 -19137693.208240177) / Math.Sqrt((483640654442.2368 / (173)) + (562433214.3845619 / (29))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (173) + (29) - 2, .975) and 0.11699968514681172 = (21673484.01390106 - 19137693.208240177) / 21673484.01390106 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w]+://[^/\\s?#]+[^\\s?#]+(?:\\?[^\\s#]*)?(?:#[^\\s]*)?", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.2475363979166665 < 2.270857588373397. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 1/13/2023 9:23:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 45.32027098418741 (T) = (0 -2244350.4563401216) / Math.Sqrt((1072403290.0379517 / (292)) + (103772409.53110185 / (21))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (292) + (21) - 2, .975) and 0.055950380674860474 = (2377364.9291278897 - 2244350.4563401216) / 2377364.9291278897 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).SquareRootBenchmark.html>) 46.95 ns 42.09 ns 0.90 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 42.088621683606206 < 44.42099746177854. IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 25.27188042725923 (T) = (0 -42.126649114955036) / Math.Sqrt((17.508256702813203 / (206)) + (0.15883430292379547 / (29))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (29) - 2, .975) and 0.15285727726244114 = (49.72792421425981 - 42.126649114955036) / 49.72792421425981 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForeachOverList100Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.LowLevelPerf.ForeachOverList100Elements.html>) 14.38 ms 10.91 ms 0.76 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ForeachOverList100Elements ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.911152258333331 < 13.661847545138887. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 649.3122010002667 (T) = (0 -10911512.708702177) / Math.Sqrt((8732794387.937208 / (299)) + (1084497.986949032 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.24346071161658128 = (14422929.35244383 - 10911512.708702177) / 14422929.35244383 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConvertTryFromBase64Chars - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Base64Tests.ConvertTryFromBase64Chars(NumberOfBytes%3a%201000).html>) 1.09 μs 968.32 ns 0.89 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Base64Tests.ConvertTryFromBase64Chars(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 968.3199906367043 < 1.0258760206496529. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 178.03585242510482 (T) = (0 -970.9682417408621) / Math.Sqrt((31.25451089171036 / (299)) + (7.572570456067193 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09980944017572062 = (1078.6252212313789 - 970.9682417408621) / 1078.6252212313789 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 348.64 ns 184.44 ns 0.53 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 184.44436265097724 < 332.09649548660406. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 39.03430068844011 (T) = (0 -205.94400732916358) / Math.Sqrt((0.13037304732800886 / (299)) + (392.21062289686796 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.41074272221799013 = (349.49760502635763 - 205.94400732916358) / 349.49760502635763 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.Tests.Add_Remove_SteadyState<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Tests.Add_Remove_SteadyState(String).Stack(Count%3a%20512).html>) 10.95 ns 9.71 ns 0.89 0.16 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Add_Remove_SteadyState<String>.Stack(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.714433287398395 < 10.405319517481525. IsChangePoint: Marked as a change because one of 10/13/2022 6:03:38 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 21.63044796075071 (T) = (0 -9.75086531374737) / Math.Sqrt((0.12312191532510942 / (299)) + (0.056092773433175105 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09702088400846359 = (10.798550200178456 - 9.75086531374737) / 10.798550200178456 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.IterateForEach(Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 8.08 μs 7.55 μs 0.93 0.01 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.IterateForEach(Int32).List(Size%3a%20512).html>) 857.71 ns 686.21 ns 0.80 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.54693777979066 < 7.635857797760257. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 54.026328900913995 (T) = (0 -7573.783204214917) / Math.Sqrt((3771.8128991342805 / (299)) + (2080.8472110520966 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.061492147728768015 = (8070.026463695551 - 7573.783204214917) / 8070.026463695551 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 686.2134165210426 < 814.8286756575644. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 2127.7798557056426 (T) = (0 -686.5812291315723) / Math.Sqrt((0.22498326492963863 / (299)) + (0.1659403490200195 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.19959468362521499 = (857.7919400151569 - 686.5812291315723) / 857.7919400151569 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Base64DecodeInPlace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes%3a%20200000000).html>) 185.71 ms 168.00 ms 0.90 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes: 200000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 168.00393640000001 < 176.38886471964284. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 289.06284643937295 (T) = (0 -167889570.7260326) / Math.Sqrt((1123969661574.5327 / (299)) + (5077943867.59523 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09746742933430506 = (186020511.81621036 - 167889570.7260326) / 186020511.81621036 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Replace_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_String.Replace_Char(text%3a%20%22yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi79.html>) 242.35 ns 206.03 ns 0.85 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 206.03282354237427 < 230.83592640013876. IsChangePoint: Marked as a change because one of 10/26/2022 10:56:11 AM, 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 28.517967805103925 (T) = (0 -211.19396029954987) / Math.Sqrt((53.69016779340178 / (289)) + (48.005101947986816 / (29))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (29) - 2, .975) and 0.15485048073606533 = (249.8894639181536 - 211.19396029954987) / 249.8894639181536 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsyncThenReadAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Threading.Channels.Tests.UnboundedChannelPerfTests.WriteAsyncThenReadAsync.html>) 96.46 ns 88.44 ns 0.92 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Channels.Tests.UnboundedChannelPerfTests.WriteAsyncThenReadAsync ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 88.44096454553693 < 91.72749083114557. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 22.225599451777946 (T) = (0 -89.07681544648267) / Math.Sqrt((4.430194329693436 / (299)) + (2.714800335513024 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07699507404242577 = (96.50741067721785 - 89.07681544648267) / 96.50741067721785 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Concurrent.Count(String).Queue(Size%3a%20512).html>) 16.87 ns 14.55 ns 0.86 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.553119142564723 < 15.993458012840922. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM. IsImprovementStdDev: Marked as improvement because 91.08529505046894 (T) = (0 -14.697549599646687) / Math.Sqrt((0.03013473374526916 / (299)) + (0.013385258351341401 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.12813152557485732 = (16.857530729434117 - 14.697549599646687) / 16.857530729434117 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)