search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 20 Improvements on 1/11/2023 1:25:06 PM #11601

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 98.37 ns 91.37 ns 0.93 0.02 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 97.54 ns 90.91 ns 0.93 0.03 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 108.51 ns 92.02 ns 0.85 0.03 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 91.36618237840015 < 92.57612070438783. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 76.44700018506414 (T) = (0 -91.3677992151856) / Math.Sqrt((2.3472984930626146 / (299)) + (5.228322381534788E-06 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.06902792234498527 = (98.1423625994541 - 91.3677992151856) / 98.1423625994541 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.90806462568844 < 92.74441051207975. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 8.818520768407037 (T) = (0 -91.6516182261652) / Math.Sqrt((3.301980345274103 / (299)) + (1.105743913563923 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.0673852509006874 = (98.27382455046866 - 91.6516182261652) / 98.27382455046866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 92.01806901728982 < 94.20259323656437. IsChangePoint: Marked as a change because one of 12/27/2022 1:08:06 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 16.001832449973204 (T) = (0 -91.61206614529866) / Math.Sqrt((4.035066308615414 / (299)) + (0.32967666413014507 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.06869505292483688 = (98.36956888612436 - 91.61206614529866) / 98.36956888612436 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).SortedDictionary(Size%3a%20512).html>) 11.41 μs 10.59 μs 0.93 0.01 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).SortedSet(Size%3a%20512).html>) 10.45 μs 9.85 μs 0.94 0.00 False
[List - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).List(Size%3a%20512).html>) 657.53 ns 494.02 ns 0.75 0.01 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.589339826839828 < 10.855083286697981. IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 32.31437216442132 (T) = (0 -10557.571433164732) / Math.Sqrt((59140.809495359994 / (299)) + (2018.4616733915757 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.09611741032168354 = (11680.246476394766 - 10557.571433164732) / 11680.246476394766 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.853000401147526 < 9.927923274254438. IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 12/7/2022 11:17:41 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 101.5816547380694 (T) = (0 -9849.08427701526) / Math.Sqrt((13460.959138111797 / (299)) + (30.67205643860854 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.07418304167520674 = (10638.262983253784 - 9849.08427701526) / 10638.262983253784 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 494.02291983799387 < 624.6677915109323. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 11/4/2022 9:11:54 AM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 67.55630332178416 (T) = (0 -494.1170715760081) / Math.Sqrt((1340.4452909697102 / (299)) + (0.017729099542190804 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.22466816350331045 = (637.2975393460679 - 494.1170715760081) / 637.2975393460679 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetMethod4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod4.html>) 339.37 μs 316.17 μs 0.93 0.00 False
[GetMethod10 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod10.html>) 843.66 μs 794.22 μs 0.94 0.00 False
[GetMethod2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod2.html>) 168.13 μs 156.95 μs 0.93 0.00 False
[GetMethod3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod3.html>) 254.07 μs 237.15 μs 0.93 0.00 False
[GetMethod12 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod12.html>) 1.01 ms 952.21 μs 0.94 0.00 False
[GetMethod5 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.GetMember.GetMethod5.html>) 419.45 μs 396.05 μs 0.94 0.00 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.GetMember.GetMethod4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 316.1693035714286 < 322.7132230525362. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 58.82507701241798 (T) = (0 -316509.6059523809) / Math.Sqrt((9223179.589288805 / (299)) + (231611.42096370787 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.06644469952709528 = (339036.8045599964 - 316509.6059523809) / 339036.8045599964 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod10 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 794.2233333333334 < 803.7631354166666. IsChangePoint: Marked as a change because one of 9/29/2022 6:03:53 PM, 10/6/2022 9:08:45 AM, 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 28.354588803826655 (T) = (0 -796083.5639880953) / Math.Sqrt((75173660.9447533 / (299)) + (6920916.177831727 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.06421513294239108 = (850712.1583310309 - 796083.5639880953) / 850712.1583310309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 156.95467916666666 < 160.10612399193548. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 11/2/2022 5:57:48 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 26.92777586203624 (T) = (0 -157400.72552609426) / Math.Sqrt((2518320.8317912524 / (299)) + (397914.7095172197 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.07227626186779522 = (169663.35888202093 - 157400.72552609426) / 169663.35888202093 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 237.14629353233835 < 240.18097977470555. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 56.45946065988908 (T) = (0 -236870.21116010856) / Math.Sqrt((5861239.662705289 / (299)) + (152442.95251202857 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.06871428205518045 = (254347.51827058897 - 236870.21116010856) / 254347.51827058897 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod12 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 952.2080357142856 < 0.962249125708276. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 11/2/2022 5:57:48 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 59.696870653289274 (T) = (0 -951240.7169117646) / Math.Sqrt((117505400.27520803 / (299)) + (1871411.3314214782 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.067459389550081 = (1020052.8601674767 - 951240.7169117646) / 1020052.8601674767 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.GetMember.GetMethod5 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 396.0470208333333 < 398.76824915540533. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 11/2/2022 9:17:08 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 20.83971213452364 (T) = (0 -394712.17628205125) / Math.Sqrt((17475817.41140201 / (299)) + (3563619.952174662 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.06683571545714038 = (422982.5153192759 - 394712.17628205125) / 422982.5153192759 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseDecimal - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value%3a%20123456.789).html>) 49.58 ns 46.18 ns 0.93 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 46.17598555930514 < 47.17827439765098. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 100.94637010117606 (T) = (0 -46.20276162532868) / Math.Sqrt((0.1855914325265376 / (299)) + (0.001433915423393981 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.07399616527671396 = (49.894784333301665 - 46.20276162532868) / 49.894784333301665 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsyncThenReadAsync - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync.html>) 80.94 ns 73.03 ns 0.90 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 73.0250546159522 < 76.83726259917526. IsChangePoint: Marked as a change because one of 10/20/2022 5:12:08 PM, 10/26/2022 2:40:44 AM, 11/2/2022 9:17:08 PM, 12/5/2022 9:44:53 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 9.649641867835738 (T) = (0 -72.2168131991194) / Math.Sqrt((2.1297573680244253 / (299)) + (1.306508375767761 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.09794899412052761 = (80.0584586995834 - 72.2168131991194) / 80.0584586995834 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForEach_Array - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array.html>) 5.38 ns 2.80 ns 0.52 0.17 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.797530406804249 < 5.206950221753115. IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/6/2022 9:08:45 AM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 28.077423695321365 (T) = (0 -2.7024567048642743) / Math.Sqrt((0.32691706115929636 / (299)) + (0.018078017601142242 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.5111968532354335 = (5.528721987065931 - 2.7024567048642743) / 5.528721987065931 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Replace_Char - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Replace_Char(text%3a%20%22yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi79.html>) 255.42 ns 233.46 ns 0.91 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 233.4588172638091 < 242.5268792775347. IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 11/11/2022 5:53:56 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 19.35071961918021 (T) = (0 -234.636370874427) / Math.Sqrt((28.40402280838542 / (299)) + (2.7732650117585345 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.09122768330598559 = (258.19049124207595 - 234.636370874427) / 258.19049124207595 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableHashSet(Size%3a%20512).html>) 272.72 μs 238.42 μs 0.87 0.19 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 238.41695075757573 < 250.20495835813495. IsChangePoint: Marked as a change because one of 11/30/2022 3:45:04 PM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 28.200004434834483 (T) = (0 -239423.9012029637) / Math.Sqrt((114496930.44370933 / (299)) + (2027898.3989339778 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.12219652161261349 = (272753.4204384899 - 239423.9012029637) / 272753.4204384899 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 332.88 ns 210.09 ns 0.63 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 210.08876579713558 < 316.2929590181982. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 4319.609130536946 (T) = (0 -210.07990854818945) / Math.Sqrt((0.21827446183759924 / (299)) + (0.0001569017177878921 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.3689430353623065 = (332.9016559841026 - 210.07990854818945) / 332.9016559841026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForeachOverList100Elements - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.LowLevelPerf.ForeachOverList100Elements.html>) 13.67 ms 9.64 ms 0.71 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ForeachOverList100Elements ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.640575757575755 < 12.98640405885781. IsChangePoint: Marked as a change because one of 11/4/2022 9:11:54 AM, 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 44.68238024215537 (T) = (0 -9645938.927738927) / Math.Sqrt((1687617042460.202 / (299)) + (57527188.398243174 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.2586534256033125 = (13011375.867742902 - 9645938.927738927) / 13011375.867742902 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 7ac3cde6be0d10e0bb8d6e5fe0b95eeca549e2b8
Compare 2ca7cf7140ebfaa8e34732b529c194416b122e89
Diff Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IndexerSet(Int32).ConcurrentDictionary(Size%3a%20512).html>) 26.44 μs 19.10 μs 0.72 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.097428039452495 < 25.068276429368993. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/12/2023 6:14:36 AM falls between 1/3/2023 3:01:43 PM and 1/12/2023 6:14:36 AM. IsImprovementStdDev: Marked as improvement because 76.0089162678131 (T) = (0 -19000.91880646493) / Math.Sqrt((78854.55476090226 / (299)) + (18628.064103695215 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.28134646922079926 = (26439.609620874147 - 19000.91880646493) / 26439.609620874147 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)