search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 27 Improvements on 8/17/2022 3:53:31 PM #7385

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbb.html>) 31.41 ns 26.19 ns 0.83 0.03 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbbb.html>) 15.79 ns 12.05 ns 0.76 0.03 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%22%2c%20value%3a%20%22x%22%2c%20comparisonType%3a%20OrdinalIgnoreCase).html>) 30.28 ns 20.56 ns 0.68 0.01 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22StrIng%22%2c%20value%3a%20%22string%22%2c%20comparisonType%3a%20OrdinalIgnoreCase).html>) 18.24 ns 15.68 ns 0.86 0.01 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%99%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%22%2c%20value%3a%20%22%e3%81%99%22%2c%20comparisonType%3a%20Ordinal).html>) 17.80 ns 13.71 ns 0.77 0.02 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!%22%2c%20value%3a%20%22y%22%2c%20comparisonType%3a%20Ordinal).html>) 17.36 ns 11.62 ns 0.67 0.01 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 38.924712649774854 (T) = (0 -26.283769301579778) / Math.Sqrt((7.810436262209558 / (299)) + (0.01208142912834035 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.20179615354250496 = (32.92864274988107 - 26.283769301579778) / 32.92864274988107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/17/2022 3:53:31 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 41.849674547721094 (T) = (0 -12.007440727924783) / Math.Sqrt((3.0015019454526146 / (299)) + (0.004923851547673028 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2736083217770292 = (16.530256455166903 - 12.007440727924783) / 16.530256455166903 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/13/2022 3:46:07 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 92.71311882232149 (T) = (0 -20.55737460986397) / Math.Sqrt((3.456874360442512 / (299)) + (4.20214646951011E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3266678114700637 = (30.530806279655515 - 20.55737460986397) / 30.530806279655515 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/17/2022 3:53:31 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 44.55584096986423 (T) = (0 -15.6760362710066) / Math.Sqrt((1.1580826273806937 / (299)) + (5.2867942681125245E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15059281587875484 = (18.455266877951185 - 15.6760362710066) / 18.455266877951185 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 6:57:51 AM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 49.36972890075971 (T) = (0 -13.648420020060636) / Math.Sqrt((1.1936072375916171 / (299)) + (0.0018600981410580948 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.19451684396616578 = (16.94438911331789 - 13.648420020060636) / 16.94438911331789 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 129.24935187827057 (T) = (0 -11.618624859218194) / Math.Sqrt((0.5096425029666476 / (299)) + (0.0016113124219285094 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.33804120856464687 = (17.551885418766084 - 11.618624859218194) / 17.551885418766084 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Span(Int32).LastIndexOfValue(Size%3a%20512).html>) 89.39 ns 52.59 ns 0.59 0.01 False
[IndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Span(Int32).IndexOfValue(Size%3a%20512).html>) 58.63 ns 52.34 ns 0.89 0.04 False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Span(Int32).Reverse(Size%3a%20512).html>) 171.90 ns 93.59 ns 0.54 0.03 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.LastIndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 111.21281367406968 (T) = (0 -51.88718303485445) / Math.Sqrt((3.613095573925244 / (299)) + (0.4148307554156321 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.42174654722043375 = (89.73086591258827 - 51.88718303485445) / 89.73086591258827 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Int32>.IndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 32.89654490331339 (T) = (0 -52.451616028256176) / Math.Sqrt((4.89809069042654 / (299)) + (0.06232513615064451 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.10082295651014536 = (58.33291275395865 - 52.451616028256176) / 58.33291275395865 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Int32>.Reverse(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 33.024148013865 (T) = (0 -97.50269285636779) / Math.Sqrt((3205.0585380844777 / (299)) + (68.88973176789618 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.545736453909847 = (214.63904311841355 - 97.50269285636779) / 214.63904311841355 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Contains_ElementNotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input%3a%20ICollection).html>) 27.52 ns 24.04 ns 0.87 0.04 False
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToArray(input%3a%20IEnumerable).html>) 1.12 μs 1.06 μs 0.95 0.03 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 107.86877884930209 (T) = (0 -24.04452205044999) / Math.Sqrt((0.24861773869269446 / (299)) + (0.00020383885651708485 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.11759632247050075 = (27.24889147988186 - 24.04452205044999) / 27.24889147988186 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 11.683618868982608 (T) = (0 -1054.0803364393641) / Math.Sqrt((310.5192460414534 / (299)) + (87.87294049885894 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05753118598782294 = (1118.4246319536521 - 1054.0803364393641) / 1118.4246319536521 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).ImmutableArray(Size%3a%20512).html>) 61.99 μs 57.48 μs 0.93 0.01 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).Span(Size%3a%20512).html>) 60.66 μs 50.83 μs 0.84 0.01 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).Array(Size%3a%20512).html>) 62.88 μs 55.43 μs 0.88 0.01 False
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).ICollection(Size%3a%20512).html>) 62.59 μs 56.02 μs 0.90 0.01 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).Stack(Size%3a%20512).html>) 97.30 μs 56.01 μs 0.58 0.00 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).Queue(Size%3a%20512).html>) 63.36 μs 58.87 μs 0.93 0.01 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).List(Size%3a%20512).html>) 62.11 μs 54.71 μs 0.88 0.00 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 27.479202713200507 (T) = (0 -57264.0002462809) / Math.Sqrt((1173905.7212627924 / (299)) + (158420.27549033068 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.09100859628009556 = (62997.29569711768 - 57264.0002462809) / 62997.29569711768 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 117.16897324065687 (T) = (0 -50853.366642579196) / Math.Sqrt((1341463.6635787415 / (299)) + (5777.605808702215 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.15070034189967582 = (59876.82457841294 - 50853.366642579196) / 59876.82457841294 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 26.22082965506751 (T) = (0 -55836.96059526298) / Math.Sqrt((1052076.0548349817 / (299)) + (201778.86201954525 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.09835771679725593 = (61928.06352972183 - 55836.96059526298) / 61928.06352972183 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 25.620192529967337 (T) = (0 -55625.90089073121) / Math.Sqrt((1038901.6252620941 / (299)) + (210201.25906852484 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.09830090547566692 = (61690.092879681935 - 55625.90089073121) / 61690.092879681935 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 429.4161671424927 (T) = (0 -55918.21567541177) / Math.Sqrt((1221.3002280528926 / (299)) + (37152.45375933784 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.4253732383558749 = (97312.23710399125 - 55918.21567541177) / 97312.23710399125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 22.843126666779284 (T) = (0 -58633.89419315485) / Math.Sqrt((1152394.375101021 / (299)) + (233438.55236905508 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.0885675553154501 = (64331.585445642384 - 58633.89419315485) / 64331.585445642384 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 31.778025367823513 (T) = (0 -54891.91208167003) / Math.Sqrt((1165141.0752155953 / (299)) + (134101.1534521906 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.10071201976520827 = (61039.303635903714 - 54891.91208167003) / 61039.303635903714 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).Queue(Size%3a%20512).html>) 42.50 μs 33.34 μs 0.78 0.00 False
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).ICollection(Size%3a%20512).html>) 40.15 μs 32.41 μs 0.81 0.00 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).Stack(Size%3a%20512).html>) 50.48 μs 31.01 μs 0.61 0.01 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).ImmutableArray(Size%3a%20512).html>) 42.72 μs 34.43 μs 0.81 0.00 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).List(Size%3a%20512).html>) 38.97 μs 31.73 μs 0.81 0.00 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).Span(Size%3a%20512).html>) 36.30 μs 28.56 μs 0.79 0.00 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).Array(Size%3a%20512).html>) 42.17 μs 32.46 μs 0.77 0.00 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 64.19162867990582 (T) = (0 -33250.43820726043) / Math.Sqrt((363944.6434769154 / (299)) + (68766.5002658131 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.20756638425293247 = (41959.903702360665 - 33250.43820726043) / 41959.903702360665 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 52.420624251132416 (T) = (0 -32618.869140262024) / Math.Sqrt((370539.5137753595 / (299)) + (96239.6001430945 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.20357753937915776 = (40956.74187143636 - 32618.869140262024) / 40956.74187143636 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 45.09421011648532 (T) = (0 -31479.730686959432) / Math.Sqrt((8066.971157684461 / (299)) + (711735.9420824404 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3766749465128667 = (50502.91258284749 - 31479.730686959432) / 50502.91258284749 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 40.64209510440087 (T) = (0 -34116.83433475343) / Math.Sqrt((358720.99800460675 / (299)) + (158287.93248533088 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.1938992743980479 = (42323.28944906586 - 34116.83433475343) / 42323.28944906586 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 220.0891111685754 (T) = (0 -31750.682894269077) / Math.Sqrt((352458.17450652586 / (299)) + (593.570707717494 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.2016151287882368 = (39768.64296799475 - 31750.682894269077) / 39768.64296799475 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 25.831719054204328 (T) = (0 -28858.69604651744) / Math.Sqrt((399694.7048172157 / (299)) + (419488.530503967 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.22583473902999548 = (37277.17775705721 - 28858.69604651744) / 37277.17775705721 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 37.286548445922115 (T) = (0 -32680.950413352624) / Math.Sqrt((325039.3122585401 / (299)) + (228259.49672767092 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.21576691503894938 = (41672.49640452971 - 32680.950413352624) / 41672.49640452971 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CopyToOverwrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_File.CopyToOverwrite(size%3a%20104857600).html>) 68.45 ms 61.45 ms 0.90 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 19.02508574262159 (T) = (0 -62054977.76098901) / Math.Sqrt((5595211405262.925 / (299)) + (254773678978.7294 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.08089058074802548 = (67516420.20108227 - 62054977.76098901) / 67516420.20108227 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryReadTo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.SequenceReader.TryReadTo.html>) 260.62 ns 166.15 ns 0.64 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.SequenceReader.TryReadTo ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM. IsImprovementStdDev: Marked as improvement because 246.4856982671677 (T) = (0 -165.79693042628878) / Math.Sqrt((10.968228997845358 / (299)) + (0.45722609338850423 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.36615863620830014 = (261.57480388227685 - 165.79693042628878) / 261.57480388227685 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)