search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Linux/x64: 20 Improvements on 10/2/2022 4:42:49 PM #8245

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CreateAddAndClear(String).ICollection(Size%3a%20512).html>) 3.08 μs 2.74 μs 0.89 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.CreateAddAndClear-String-.ICollection(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.CreateAddAndClear<String>.ICollection(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7361323774080564 < 2.9224469611147.
IsChangePoint: Marked as a change because one of 8/5/2022 4:34:47 AM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 24.683893319440998 (T) = (0 -2785.502496980849) / Math.Sqrt((17216.629390632716 / (299)) + (734.6879872264814 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07735915067932614 = (3019.053946106734 - 2785.502496980849) / 3019.053946106734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Char_ToLowerInvariant - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Char.Char_ToLowerInvariant(input%3a%20%22Hello%20World!%22).html>) 17.99 ns 16.27 ns 0.90 0.02 False
[Char_ToUpperInvariant - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Char.Char_ToUpperInvariant(input%3a%20%22Hello%20World!%22).html>) 18.06 ns 16.50 ns 0.91 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Char.Char_ToUpperInvariant(input- -Hello World!-).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.26896703931754 < 16.889889073957384.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 33.76960430510867 (T) = (0 -16.245823783463305) / Math.Sqrt((0.15212012788994972 / (299)) + (0.028508637600778385 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07996377414852299 = (17.6578088198952 - 16.245823783463305) / 17.6578088198952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToUpperInvariant(input: "Hello World!")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.498165349485355 < 17.019352165605838.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 28.53542847260923 (T) = (0 -16.522601625452566) / Math.Sqrt((0.1291608682683608 / (299)) + (0.03070916239505038 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0666846599311882 = (17.70312874556887 - 16.522601625452566) / 17.70312874556887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetUInt64 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetUInt64.html>) 537.82 ns 490.97 ns 0.91 0.02 False
[GetString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetString.html>) 3.06 μs 2.78 μs 0.91 0.08 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetString.txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Text.Json.Tests.Perf_Get.GetUInt64

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 490.97031092688144 < 510.89254820509524.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 21.204243961148936 (T) = (0 -491.4564244880077) / Math.Sqrt((787.885120009058 / (299)) + (1.8304240728698888 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.0663696795262824 = (526.392956302711 - 491.4564244880077) / 526.392956302711 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.775759488597685 < 2.922075074492296.
IsChangePoint: Marked as a change because one of 8/18/2022 3:00:51 AM, 9/8/2022 6:17:28 PM, 9/13/2022 12:23:36 PM, 10/3/2022 8:45:37 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 17.7354165686935 (T) = (0 -2811.9075113541007) / Math.Sqrt((6637.425682999178 / (299)) + (2366.6468661047506 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.08022622812513225 = (3057.172967242049 - 2811.9075113541007) / 3057.172967242049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(Int32).ImmutableStack(Size%3a%20512).html>) 1.47 μs 1.25 μs 0.85 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach-Int32-.ImmutableStack(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.IterateForEach<Int32>.ImmutableStack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.24928508337997 < 1.3156878781921821.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 9/3/2022 11:44:47 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 76.66611107336118 (T) = (0 -1250.1354882046965) / Math.Sqrt((1370.2980752667393 / (299)) + (4.30967656888747 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.118214354549531 = (1417.7317295362097 - 1250.1354882046965) / 1417.7317295362097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(String).ImmutableStack(Size%3a%20512).html>) 1.61 μs 1.46 μs 0.91 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach-String-.ImmutableStack(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.IterateForEach<String>.ImmutableStack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.461672080913122 < 1.5267418757807885.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 74.82997779001889 (T) = (0 -1463.7340547382128) / Math.Sqrt((811.2340872812667 / (299)) + (8.643352933828057 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08244274335969329 = (1595.250916654255 - 1463.7340547382128) / 1595.250916654255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NextBytes_span - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_span.html>) 6.19 μs 5.68 μs 0.92 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_span.txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Tests.Perf_Random.NextBytes_span

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.676347324346405 < 5.875967569021087.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 69.2093033454498 (T) = (0 -5675.244533242368) / Math.Sqrt((24675.102187351735 / (299)) + (6.645012183248313 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09988475701947637 = (6305.019915505604 - 5675.244533242368) / 6305.019915505604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexerCheckPathLength - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.IndexerCheckPathLength.html>) 96.30 ns 82.26 ns 0.85 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.IndexerCheckPathLength.txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Tests.Perf_String.IndexerCheckPathLength

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.25521047292165 < 91.91033392533103.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 162.31221464293571 (T) = (0 -82.41439821416654) / Math.Sqrt((1.0020706980982743 / (299)) + (0.1186392153922924 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15374613646465068 = (97.387322841716 - 82.41439821416654) / 97.387322841716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22r%22).html>) 37.53 ns 34.00 ns 0.91 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_DateTimeOffset.ToString(format- -r-).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Tests.Perf_DateTimeOffset.ToString(format: "r")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.00447258873488 < 35.699163775441804.
IsChangePoint: Marked as a change because one of 7/31/2022 12:39:41 PM, 8/23/2022 7:19:33 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 26.522026335546204 (T) = (0 -33.99318704373916) / Math.Sqrt((2.8887381509546786 / (299)) + (0.12135953403854657 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08646931982428223 = (37.21077767984822 - 33.99318704373916) / 37.21077767984822 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.CtorFromCollectionNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollectionNonGeneric(String).Stack(Size%3a%20512).html>) 4.97 μs 4.48 μs 0.90 0.01 False
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollectionNonGeneric(String).Queue(Size%3a%20512).html>) 5.70 μs 5.21 μs 0.91 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollectionNonGeneric-String-.Queue(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.CtorFromCollectionNonGeneric<String>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.482559753043719 < 4.716067158624251.
IsChangePoint: Marked as a change because one of 8/15/2022 5:23:15 PM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 35.27532490440759 (T) = (0 -4480.937784635198) / Math.Sqrt((23236.480106881256 / (299)) + (408.1979464863344 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07142247525497285 = (4825.593625977102 - 4480.937784635198) / 4825.593625977102 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.205421872915277 < 5.407056515748736.
IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 10/2/2022 11:57:04 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 32.35519096444521 (T) = (0 -5188.357628585326) / Math.Sqrt((12959.482839266908 / (299)) + (2153.064286691214 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07004219342197948 = (5579.132291686439 - 5188.357628585326) / 5579.132291686439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MatchesSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options%3a%20Compiled).html>) 43.51 μs 40.61 μs 0.93 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options- Compiled).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.6120185501998 < 41.33574212532159.
IsChangePoint: Marked as a change because one of 7/26/2022 1:50:03 PM, 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 147.74120751216327 (T) = (0 -40568.499415512364) / Math.Sqrt((63477.450929994586 / (299)) + (4634.615577440341 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06897200977252521 = (43573.87730695444 - 40568.499415512364) / 43573.87730695444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor(String).ImmutableSortedSet(Size%3a%20512).html>) 12.07 μs 11.31 μs 0.94 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor-String-.ImmutableSortedSet(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.314187341354451 < 11.466214740828121.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/1/2022 8:26:50 AM, 8/4/2022 6:25:19 PM, 8/15/2022 9:13:40 PM, 9/23/2022 1:16:35 AM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 22.74671912110157 (T) = (0 -11149.14183699705) / Math.Sqrt((136257.1502983908 / (299)) + (11318.932433274036 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05849484363815016 = (11841.827696493345 - 11149.14183699705) / 11841.827696493345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.40).html>) 4.78 ns 3.66 ns 0.77 0.05 False
[Call - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.50).html>) 5.10 ns 3.76 ns 0.74 0.05 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/GuardedDevirtualization.TwoClassVirtual.Call(testInput- pB = 0.50).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.40)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.661317775356809 < 4.471206199816343.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/1/2022 8:04:48 PM, 9/27/2022 11:22:50 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 16.64673071628283 (T) = (0 -3.728792532604513) / Math.Sqrt((0.07862039180697353 / (299)) + (0.01399799672813818 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11644982833164415 = (4.220238592182777 - 3.728792532604513) / 4.220238592182777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.50)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7576366745755245 < 4.844797876570347.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/1/2022 8:04:48 PM, 9/27/2022 11:22:50 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 21.385781179682844 (T) = (0 -3.823900777899534) / Math.Sqrt((0.15562734488840743 / (299)) + (0.014031852820850671 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1582829084607764 = (4.5429762759205445 - 3.823900777899534) / 4.5429762759205445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ScaleB - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.ScaleB.html>) 11.94 μs 10.08 μs 0.84 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.ScaleB.txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.MathBenchmarks.Single.ScaleB

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.08192372321827 < 11.345361018996982.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 346.4394217680892 (T) = (0 -10077.708985170064) / Math.Sqrt((8504.75560549852 / (299)) + (28.465345845254006 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1576286960299877 = (11963.499869564434 - 10077.708985170064) / 11963.499869564434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayByteArrayCtor - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size%3a%20512).html>) 217.83 ns 178.28 ns 0.82 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 178.27948269812916 < 205.7079308861163.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 15.45659709885489 (T) = (0 -177.97927363011996) / Math.Sqrt((696.30557102586 / (299)) + (3.4858495633016107 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12031681882809504 = (202.32201483381527 - 177.97927363011996) / 202.32201483381527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEachNonGeneric(Int32).Stack(Size%3a%20512).html>) 3.29 μs 2.93 μs 0.89 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEachNonGeneric-Int32-.Stack(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.IterateForEachNonGeneric<Int32>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.931391850079156 < 3.128050720128182.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/6/2022 4:25:34 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 71.18085626988959 (T) = (0 -2931.268388365634) / Math.Sqrt((6981.2698260225625 / (299)) + (9.46380669228147 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10583957755997928 = (3278.235442770629 - 2931.268388365634) / 3278.235442770629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Improvements in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEachNonGeneric(String).Stack(Size%3a%20512).html>) 3.25 μs 2.93 μs 0.90 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'

Related Issues

Regressions

Payloads

Baseline Compare

Histogram

Edge Detector Info

[Collection Data](<https://pvscmdupload.blob.core.windows.net/autofilereport/reports/10_06_2022/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEachNonGeneric-String-.Stack(Size- 512).txt?sv=2018-03-28&sr=c&sig=LG%2FuZBS96wm3PV3z1k%2FG4jh6a9y8FAY3G0pihsGACTQ%3D&se=2022-10-23T12%3A15%3A03Z&sp=racwdl>)

System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9268934448929778 < 3.090272212644492.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/9/2022 6:41:10 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsImprovementStdDev: Marked as improvement because 66.93508968135524 (T) = (0 -2931.076163448584) / Math.Sqrt((8048.662919146422 / (299)) + (7.409439038974951 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10649589440666371 = (3280.42831040176 - 2931.076163448584) / 3280.42831040176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository