search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 5/23/2022 8:22:56 AM #5552

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(CollectionsOfPrimitives).DataContractJsonSerializer.html>) 4.84 ms 5.51 ms 1.14 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -9.20350057203344 (T) = (0 -5701102.10708459) / Math.Sqrt((3245561071.164633 / (3)) + (88589970342.37424 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.17442859331955288 = (4854362.487011898 - 5701102.10708459) / 4854362.487011898 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20None%2c%20False)).html>) 7.57 ns 13.40 ns 1.77 0.25 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 5.68 ns 9.99 ns 1.76 0.52 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 7.54 ns 12.93 ns 1.72 0.24 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 8.29 ns 13.79 ns 1.66 0.26 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 8.01 ns 13.38 ns 1.67 0.22 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 8.35 ns 13.83 ns 1.66 0.23 True

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -36.8652753015391 (T) = (0 -13.518154036460002) / Math.Sqrt((0.03068291652988026 / (3)) + (0.19997637576713662 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.8090148506579522 = (7.472660620526884 - 13.518154036460002) / 7.472660620526884 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -5.386375579424582 (T) = (0 -10.086074836487223) / Math.Sqrt((1.2438983502193335 / (3)) + (0.04287372585149476 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.5275521751765162 = (6.60276945062235 - 10.086074836487223) / 6.60276945062235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -78.76446900894463 (T) = (0 -13.243974366781034) / Math.Sqrt((7.449482592399096E-05 / (3)) + (0.0627012918243052 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.7571992241516591 = (7.536979407201231 - 13.243974366781034) / 7.536979407201231 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -135.62356393825073 (T) = (0 -13.821405939723322) / Math.Sqrt((0.004256845767637584 / (3)) + (0.003382175283103873 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.6797619471144674 = (8.228193264805196 - 13.821405939723322) / 8.228193264805196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -38.470287814336174 (T) = (0 -13.359809652185822) / Math.Sqrt((0.03621969671286031 / (3)) + (0.10654919081370137 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.714716109560448 = (7.791266191352508 - 13.359809652185822) / 7.791266191352508 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -62.62924907234934 (T) = (0 -13.83121595363965) / Math.Sqrt((0.008028814966667451 / (3)) + (0.0631752260372005 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.6764653908751421 = (8.250224567069369 - 13.83121595363965) / 8.250224567069369 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Utf8Encoding.GetString(Input%3a%20Cyrillic).html>) 217.92 μs 261.25 μs 1.20 0.01 False
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Utf8Encoding.GetString(Input%3a%20Greek).html>) 341.47 μs 412.80 μs 1.21 0.00 True
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Utf8Encoding.GetString(Input%3a%20Chinese).html>) 273.42 μs 324.89 μs 1.19 0.00 True

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/13/2022 4:27:35 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -70.95027032335538 (T) = (0 -260877.77635067003) / Math.Sqrt((957289.510582211 / (3)) + (357452.954130913 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.19138598899567877 = (218969.98853460266 - 260877.77635067003) / 218969.98853460266 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Perf_Utf8Encoding.GetString(Input: Greek) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/16/2022 8:46:33 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -246.38065522834816 (T) = (0 -412814.8769660529) / Math.Sqrt((202385.92074612368 / (3)) + (182693.21171179236 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.20717569908827072 = (341967.517468944 - 412814.8769660529) / 341967.517468944 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -367.1920081892484 (T) = (0 -324887.8984022638) / Math.Sqrt((11982.208760694532 / (3)) + (188603.86211139028 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.18860448664323076 = (273335.58139241784 - 324887.8984022638) / 273335.58139241784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20True).html>) 3.75 μs 4.01 μs 1.07 0.00 True
[SingleParallel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleParallel(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 6.37 μs 6.95 μs 1.09 0.05 False
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 3.33 μs 3.61 μs 1.08 0.00 True
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20True).html>) 3.32 μs 3.58 μs 1.08 0.00 True
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 26.74 μs 28.78 μs 1.08 0.00 True
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 3.75 μs 4.02 μs 1.07 0.00 True
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20True).html>) 27.42 μs 29.48 μs 1.08 0.00 True
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20True).html>) 895.06 ns 962.05 ns 1.07 0.09 False
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20True).html>) 27.79 μs 30.02 μs 1.08 0.00 True
[SingleParallel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleParallel(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20True).html>) 6.36 μs 6.85 μs 1.08 0.02 False
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 27.20 μs 29.23 μs 1.07 0.00 True

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -59.28225006158402 (T) = (0 -4034.3211920634926) / Math.Sqrt((17.991875413333798 / (3)) + (203.84596250008534 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.07578754286634512 = (3750.1096000000002 - 4034.3211920634926) / 3750.1096000000002 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -4.823776681545296 (T) = (0 -7079.4546978973185) / Math.Sqrt((29797.303798413508 / (3)) + (23199.90232494403 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.08017306891649494 = (6554.000374216524 - 7079.4546978973185) / 6554.000374216524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -150.9807453899997 (T) = (0 -3606.79928943834) / Math.Sqrt((3.3760588914825016 / (3)) + (26.293070381839254 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.08252257385739943 = (3331.8467222222225 - 3606.79928943834) / 3331.8467222222225 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -67.13702397154916 (T) = (0 -3594.8982408730158) / Math.Sqrt((39.418272103744826 / (3)) + (43.500994950387195 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.08279768530950815 = (3320.0091666666667 - 3594.8982408730158) / 3320.0091666666667 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -106.9491145162467 (T) = (0 -28821.109224145297) / Math.Sqrt((316.2383654432763 / (3)) + (3230.4239230063667 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.07737983509389008 = (26751.1125467033 - 28821.109224145297) / 26751.1125467033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -42.099084470499285 (T) = (0 -4034.0450056776554) / Math.Sqrt((39.71646003703151 / (3)) + (367.45081760850326 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.07424517930527579 = (3755.236777777778 - 4034.0450056776554) / 3755.236777777778 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -88.15571018702396 (T) = (0 -29543.998722954824) / Math.Sqrt((313.103447290348 / (3)) + (5845.157714201602 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.07824622306118996 = (27400.04842222222 - 29543.998722954824) / 27400.04842222222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 5/9/2022 6:32:22 AM, 5/24/2022 10:45:34 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -6.635225993937266 (T) = (0 -973.6855735347986) / Math.Sqrt((360.92943260116283 / (11)) + (310.682399663361 / (4))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (4) - 2, .025) and -0.07715453962521938 = (903.942319988345 - 973.6855735347986) / 903.942319988345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -190.09641008851787 (T) = (0 -30024.209540567765) / Math.Sqrt((342.86624721296835 / (3)) + (258.4167408463086 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.0796672217812542 = (27808.76267692307 - 30024.209540567765) / 27808.76267692307 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -14.543073175282414 (T) = (0 -6891.286577241716) / Math.Sqrt((2197.4560399408556 / (3)) + (5080.316216824504 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.0772928768704901 = (6396.855233333333 - 6891.286577241716) / 6396.855233333333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -157.67368614133642 (T) = (0 -29274.318172023806) / Math.Sqrt((137.73811920846708 / (3)) + (1542.6708590210692 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.07659177641811872 = (27191.66058412698 - 29274.318172023806) / 27191.66058412698 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(Location).DataContractJsonSerializer.html>) 13.21 μs 14.58 μs 1.10 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<Location>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/18/2022 7:38:50 AM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -7.046474000746635 (T) = (0 -15084.02611506383) / Math.Sqrt((133538.06220964744 / (3)) + (181776.6471636 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.12880001295975083 = (13362.886199401271 - 15084.02611506383) / 13362.886199401271 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(MyEventsListerViewModel).DataContractJsonSerializer.html>) 4.14 ms 4.71 ms 1.14 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -7.0588140445921095 (T) = (0 -4616798.13209968) / Math.Sqrt((1342468271.4433055 / (3)) + (56307865552.46047 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.12310790403279948 = (4110734.2540479973 - 4616798.13209968) / 4110734.2540479973 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlFromStream(CollectionsOfPrimitives).DataContractSerializer.html>) 2.29 ms 2.48 ms 1.08 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -14.15142964900699 (T) = (0 -2494442.9550082036) / Math.Sqrt((415538966.9416598 / (3)) + (423533904.65033364 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.08083921069966567 = (2307876.0747340596 - 2494442.9550082036) / 2307876.0747340596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.GetService.Scoped.html>) 72.22 ns 78.19 ns 1.08 0.31 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.GetService.Scoped ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 1/4/2022 12:37:46 AM, 1/4/2022 3:12:26 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -6.00567292932264 (T) = (0 -83.62731473570359) / Math.Sqrt((0.014948322775481046 / (3)) + (43.948690417692475 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.1594579205292322 = (72.12621799809007 - 83.62731473570359) / 72.12621799809007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 3f0c2cc88dc8efdde04f422bb05ef13719ce6aa8
Compare a0ff92c8d3cb3cb3b35357117e629cdd19f7080a
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlFromStream(Location).DataContractSerializer.html>) 7.75 μs 8.30 μs 1.07 0.11 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/17/2022 8:30:53 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM. IsRegressionStdDev: Marked as regression because -4.980213147667071 (T) = (0 -8642.694606391262) / Math.Sqrt((50289.333146568926 / (3)) + (75364.34748135449 / (12))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (12) - 2, .025) and -0.09585767406804073 = (7886.694422924345 - 8642.694606391262) / 7886.694422924345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kunalspathak commented 2 years ago

Lot of regressions, but seems we are missing data. @DrewScoggins - can we do catch up run to find the culprit?

cc: @danmoseley