search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/29/2022 12:06:17 AM #5119

Closed performanceautofiler[bot] closed 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Int64).OnesComplementOperatorBenchmark.html>) 239.39 ns 400.85 ns 1.67 0.41 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Int64).OnesComplementBenchmark.html>) 241.04 ns 399.17 ns 1.66 0.41 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -30.520674892013798 (T) = (0 -409.7804651453379) / Math.Sqrt((3.3647143551696232 / (5)) + (89.9818176723673 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.7020003770285799 = (240.7640272446643 - 409.7804651453379) / 240.7640272446643 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -26.021455513005048 (T) = (0 -407.2946031744012) / Math.Sqrt((4.8411818646147085 / (5)) + (121.09323793829715 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.6970506981711465 = (240.00143520363218 - 407.2946031744012) / 240.00143520363218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LockUnlock - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_FileStream.LockUnlock(fileSize%3a%201024%2c%20options%3a%20Asynchronous).html>) 54.36 μs 83.96 μs 1.54 0.43 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileStream*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_FileStream.LockUnlock(fileSize: 1024, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -91.97057213556775 (T) = (0 -84086.65131776675) / Math.Sqrt((468706.15326705686 / (5)) + (29589.19047887183 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.5433412410954327 = (54483.512186898944 - 84086.65131776675) / 54483.512186898944 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Int16).AddBenchmark.html>) 581.28 ns 1.60 μs 2.76 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int16>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -602.7440265791046 (T) = (0 -1603.2846405547045) / Math.Sqrt((11.842510177509073 / (5)) + (1.4610201658078705 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.7418166825592647 = (584.7526753897229 - 1603.2846405547045) / 584.7526753897229 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20True)).html>) 17.23 μs 25.53 μs 1.48 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, True)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 10/7/2021 12:36:20 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -178.81003800602167 (T) = (0 -25508.666082671436) / Math.Sqrt((1566.389717220874 / (5)) + (5503.2272100444825 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.4811634478875799 = (17222.046708654358 - 25508.666082671436) / 17222.046708654358 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2012345).html>) 783.46 ns 1.41 μs 1.80 0.36 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/21/2021 1:22:16 AM, 1/23/2022 1:32:42 AM, 3/20/2022 9:41:47 PM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -85.02391490282311 (T) = (0 -1411.2656213892278) / Math.Sqrt((215.56005460163752 / (5)) + (40.009877142645166 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.826972524065565 = (772.4613275785539 - 1411.2656213892278) / 772.4613275785539 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadAllBytesAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_File.ReadAllBytesAsync(size%3a%204096).html>) 61.78 μs 91.99 μs 1.49 0.39 False
[ReadAllBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_File.ReadAllBytes(size%3a%204096).html>) 39.26 μs 64.15 μs 1.63 0.46 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 4096) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -69.01529676211827 (T) = (0 -92204.01018556701) / Math.Sqrt((385221.01797230664 / (5)) + (348476.575985275 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.4903241625951668 = (61868.42601076005 - 92204.01018556701) / 61868.42601076005 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.ReadAllBytes(size: 4096) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/24/2022 10:52:31 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -2.6390518249057506 (T) = (0 -52405.461584231765) / Math.Sqrt((515601.26807659527 / (3)) + (123740503.40753043 / (5))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (5) - 2, .025) and -0.33580604612557985 = (39231.34031039196 - 52405.461584231765) / 39231.34031039196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20Compiled).html>) 239.88 ms 409.41 ms 1.71 0.33 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22The%22%2c%20Options%3a%20Compiled).html>) 13.98 ms 32.16 ms 2.30 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -92.49384912819548 (T) = (0 -408502170.64444447) / Math.Sqrt((1768392597139.745 / (5)) + (8963846615914.215 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.70619922285765 = (239422316.6742857 - 408502170.64444447) / 239422316.6742857 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -363.89122793071516 (T) = (0 -32137737.47777778) / Math.Sqrt((11063655314.865595 / (5)) + (810940828.3017987 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.2947364064838183 = (14004979.999869281 - 32137737.47777778) / 14004979.999869281 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare_Same_Upper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20IgnoreCase)).html>) 16.96 μs 28.79 μs 1.70 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, IgnoreCase)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -37.40231570203788 (T) = (0 -29366.05309954489) / Math.Sqrt((156373.31868821377 / (5)) + (251659.5630414782 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.7612489949911376 = (16673.42504271672 - 29366.05309954489) / 16673.42504271672 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CtorInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options%3a%20Compiled).html>) 300.09 μs 475.72 μs 1.59 0.37 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -101.40408844059779 (T) = (0 -478480.6360479798) / Math.Sqrt((2397980.5204625614 / (5)) + (7956430.038573453 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.6001163807557917 = (299028.6467925392 - 478480.6360479798) / 299028.6467925392 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(SByte).EqualsAnyBenchmark.html>) 1.06 μs 2.98 μs 2.82 0.41 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -220.14995486653686 (T) = (0 -2972.2006115647196) / Math.Sqrt((103.4787157695177 / (5)) + (164.97212550938653 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.8120783415078163 = (1056.9408994384726 - 2972.2006115647196) / 1056.9408994384726 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(UInt64).AndNotBenchmark.html>) 352.59 ns 601.58 ns 1.71 0.36 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -26.751857954064953 (T) = (0 -607.9158435895562) / Math.Sqrt((172.65554773768065 / (5)) + (159.8540731512909 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.7017779330239217 = (357.22395489600626 - 607.9158435895562) / 357.22395489600626 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 17.22 μs 25.10 μs 1.46 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/4/2021 2:08:22 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -137.2741475903023 (T) = (0 -25109.813945265912) / Math.Sqrt((1985.4638010239444 / (5)) + (8801.230188058955 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.4609553178074456 = (17187.25661161897 - 25109.813945265912) / 17187.25661161897 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.XmlToStream(MyEventsListerViewModel).DataContractSerializer.html>) 40.04 ms 74.36 ms 1.86 0.35 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/5/2021 2:19:41 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -147.04293168970858 (T) = (0 -74664802.99999999) / Math.Sqrt((16538061849.810595 / (5)) + (154755824788.86755 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.8566972064030032 = (40213774.62222222 - 74664802.99999999) / 40213774.62222222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadStructAndReverseBE - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE.html>) 1.52 μs 2.48 μs 1.63 0.29 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/14/2021 1:08:06 AM, 1/21/2022 1:44:30 AM, 3/5/2022 1:15:17 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -161.17834819612472 (T) = (0 -2481.5280755414005) / Math.Sqrt((173.37237389031296 / (5)) + (3.0786962440919545 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.6342116544506421 = (1518.486340972518 - 2481.5280755414005) / 1518.486340972518 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayIntArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size%3a%20512).html>) 1.74 μs 2.41 μs 1.38 0.32 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -38.814502726094204 (T) = (0 -2408.464493269304) / Math.Sqrt((1346.1114714673852 / (5)) + (62.21062485673355 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.3782155103168098 = (1747.5238634599834 - 2408.464493269304) / 1747.5238634599834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Enumerate - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).Enumerate(Size%3a%201000).html>) 69.44 μs 238.49 μs 3.43 0.34 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -1502.0841858233869 (T) = (0 -238454.81377465127) / Math.Sqrt((15514.524978365065 / (5)) + (28736.26899414951 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -2.44086749138373 = (69300.78370404136 - 238454.81377465127) / 69300.78370404136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(UInt32).AddOperatorBenchmark.html>) 273.78 ns 677.40 ns 2.47 0.38 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(UInt32).LessThanAnyBenchmark.html>) 380.52 ns 885.16 ns 2.33 0.38 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt32>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -396.79902021774564 (T) = (0 -676.7918302546337) / Math.Sqrt((0.780445788729591 / (5)) + (2.632287585536612 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.475485676588262 = (273.3975949266629 - 676.7918302546337) / 273.3975949266629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -117.20051924982468 (T) = (0 -890.00343729848) / Math.Sqrt((46.783528368039384 / (5)) + (28.902115802603152 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.3466676761855116 = (379.2626652382126 - 890.00343729848) / 379.2626652382126 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateActivityTagObjectsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge.html>) 107.68 μs 272.17 μs 2.53 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 9/24/2021 11:12:27 PM, 10/7/2021 12:36:20 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -41.10715417974369 (T) = (0 -273377.9075100947) / Math.Sqrt((77146118.37194075 / (5)) + (4900065.5902499445 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.6393662692245656 = (103577.10132834726 - 273377.9075100947) / 103577.10132834726 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.JsonToStream(CollectionsOfPrimitives).JsonNet.html>) 21.95 ms 34.61 ms 1.58 0.29 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/5/2021 2:19:41 AM, 2/16/2022 12:49:41 AM, 2/22/2022 12:12:47 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -302.2690590438712 (T) = (0 -34584988.98095238) / Math.Sqrt((8261756564.285986 / (5)) + (336335117.11494124 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.5800874215874926 = (21888022.465367965 - 34584988.98095238) / 21888022.465367965 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SliceSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceSingleSegment.html>) 498.49 ns 907.16 ns 1.82 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/11/2021 2:11:17 AM, 2/1/2022 1:17:39 AM, 2/22/2022 12:12:47 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -160.11590791500106 (T) = (0 -905.1314401367264) / Math.Sqrt((9.848503595273346 / (5)) + (13.025132310627562 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.7998958891032286 = (502.87988634036753 - 905.1314401367264) / 502.87988634036753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[KnownSizeArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/Span.IndexerBench.KnownSizeArray(length%3a%201024).html>) 26.66 μs 104.35 μs 3.91 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Span.IndexerBench.KnownSizeArray(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/4/2021 2:08:22 AM, 1/21/2022 1:44:30 AM, 3/5/2022 1:15:17 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -885.3213008813595 (T) = (0 -104351.09458333334) / Math.Sqrt((37432.11305122607 / (5)) + (590.3803250851962 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -2.9010950863175435 = (26749.179980085035 - 104351.09458333334) / 26749.179980085035 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GreaterThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Int32).GreaterThanOrEqualBenchmark.html>) 298.62 ns 761.37 ns 2.55 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -510.9943996116715 (T) = (0 -760.58517091879) / Math.Sqrt((2.513057948918142 / (5)) + (0.9260064561925612 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.5325245906006817 = (300.32686503485803 - 760.58517091879) / 300.32686503485803 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 2.37 μs 3.52 μs 1.48 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/18/2021 1:17:23 AM, 1/21/2022 1:44:30 AM, 2/22/2022 12:12:47 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -51.21831614236746 (T) = (0 -3515.6205080000004) / Math.Sqrt((2559.2368163608135 / (5)) + (9.213572649438891 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -0.4938576631546728 = (2353.383856247619 - 3515.6205080000004) / 2353.383856247619 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 72.61 μs 235.51 μs 3.24 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 1/23/2022 1:32:42 AM, 2/22/2022 12:12:47 AM, 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -1221.4949196624757 (T) = (0 -235272.85593212701) / Math.Sqrt((9689.38926493359 / (5)) + (47279.48871858688 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -2.232922336632573 = (72774.05128673407 - 235272.85593212701) / 72774.05128673407 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 7ff3ac64bbd82144c0c9a00d09763cfeeb3f438d
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Byte).ConditionalSelectBenchmark.html>) 741.69 ns 1.76 μs 2.37 0.33 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/28/2022 1:07:28 AM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -158.67106486405933 (T) = (0 -1765.6716972872748) / Math.Sqrt((97.76969619926356 / (6)) + (75.34514405467377 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -1.3712581282174032 = (744.6138723895998 - 1765.6716972872748) / 744.6138723895998 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 686.92 μs 1.67 ms 2.43 0.34 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -610.6367847285985 (T) = (0 -1672187.669861111) / Math.Sqrt((1832956.3709368836 / (5)) + (6727888.625873735 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -1.4382274989221222 = (685821.0198188407 - 1672187.669861111) / 685821.0198188407 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare 8e624b33465546e9d05da2afea5faea233d9f847
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexerCheckPathLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.IndexerCheckPathLength.html>) 2.76 μs 8.44 μs 3.06 0.39 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.IndexerCheckPathLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/26/2022 11:56:10 PM, 5/3/2022 1:30:46 AM falls between 4/24/2022 12:27:47 AM and 5/3/2022 1:30:46 AM. IsRegressionStdDev: Marked as regression because -593.028340442142 (T) = (0 -8436.073899997396) / Math.Sqrt((457.75076744790374 / (5)) + (4.160629529285113 / (3))) is less than -2.4469118511449675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (5) + (3) - 2, .025) and -2.1025789087476454 = (2719.052165349314 - 8436.073899997396) / 2719.052165349314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)