search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 51 Improvements on 2/9/2023 4:10:34 PM #12929

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Concurrent.Count(String).Dictionary(Size%3a%20512).html>) 11.32 μs 173.38 ns 0.02 0.39 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 173.37893514411522 < 10.756070895084306. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 60.64758685577819 (T) = (0 -176.06473929865675) / Math.Sqrt((7356129.716358046 / (299)) + (6.208495100281966 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.9818280064383872 = (9688.79604220107 - 176.06473929865675) / 9688.79604220107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(String).ImmutableHashSet(Size%3a%20512).html>) 130.26 μs 117.07 μs 0.90 0.01 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(String).ConcurrentDictionary(Size%3a%20512).html>) 48.61 μs 31.69 μs 0.65 0.48 False
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(String).ImmutableDictionary(Size%3a%20512).html>) 150.53 μs 132.74 μs 0.88 0.17 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 117.06706498388829 < 122.9314538465781. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 29.815233216163193 (T) = (0 -118363.79517515448) / Math.Sqrt((3652888.7812798982 / (299)) + (2299755.3327902732 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08078552101713939 = (128766.24322337448 - 118363.79517515448) / 128766.24322337448 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 31.69385079541901 < 64.48212286922879. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 53.316399051696756 (T) = (0 -31707.051719889463) / Math.Sqrt((112991205.36091408 / (299)) + (151849.03800545624 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.5105454556582836 = (64780.38070426606 - 31707.051719889463) / 64780.38070426606 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 132.73736369377448 < 142.8728268456453. IsChangePoint: Marked as a change because one of 1/8/2023 4:08:48 AM, 2/9/2023 1:20:08 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 42.20152259165184 (T) = (0 -131241.57473626747) / Math.Sqrt((21344956.115047254 / (299)) + (956253.9942174162 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09685630295598882 = (145316.3822831528 - 131241.57473626747) / 145316.3822831528 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Concurrent.Count(Int32).Dictionary(Size%3a%20512).html>) 11.32 μs 172.29 ns 0.02 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 172.28864488695334 < 10.756992096633326. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 2122.5210847566796 (T) = (0 -173.2233757182334) / Math.Sqrt((8063.481999813465 / (299)) + (5.849753805237693 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9846021868652273 = (11249.868679536312 - 173.2233757182334) / 11249.868679536312 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Next - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.Next.html>) 7.95 ns 2.74 ns 0.34 0.07 False
[NextSingle - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextSingle.html>) 8.32 ns 3.84 ns 0.46 0.09 False
[Next_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.Next_int.html>) 9.57 ns 4.46 ns 0.47 0.08 False
[NextBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes.html>) 6.32 μs 2.27 μs 0.36 0.01 False
[Next_int_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.Next_int_int.html>) 9.63 ns 4.64 ns 0.48 0.13 False
[Next_long_long - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.Next_long_long.html>) 38.96 ns 25.81 ns 0.66 0.04 False
[NextBytes_span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_span.html>) 6.35 μs 2.27 μs 0.36 0.03 False
[NextDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextDouble.html>) 8.12 ns 3.72 ns 0.46 0.10 False
[Next_long - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.Next_long.html>) 34.70 ns 22.71 ns 0.65 0.09 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.Next ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.7366283396900557 < 7.576839256728329. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 92.19961288141361 (T) = (0 -2.811466011074358) / Math.Sqrt((0.034524001972390535 / (299)) + (0.06589979660158211 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.6465320390771404 = (7.953948651340224 - 2.811466011074358) / 7.953948651340224 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextSingle ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.843499107256905 < 7.874472411318695. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 86.77928949645008 (T) = (0 -3.9075421932338417) / Math.Sqrt((0.06312042335506854 / (299)) + (0.05138487255840093 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.5284684021132642 = (8.286914834013842 - 3.9075421932338417) / 8.286914834013842 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.Next_int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.4607859261679135 < 8.826694168699952. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 83.59249623320389 (T) = (0 -4.544779932975158) / Math.Sqrt((0.041552589666072434 / (299)) + (0.07068611360819038 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5103953705038098 = (9.282550979249926 - 4.544779932975158) / 9.282550979249926 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.2736631977358908 < 6.011146058709819. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 732.3941100674826 (T) = (0 -2277.3754419627853) / Math.Sqrt((9329.253272223668 / (299)) + (4.068839292315582 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.6430438708819398 = (6379.986940102557 - 2277.3754419627853) / 6379.986940102557 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.Next_int_int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.644045358692717 < 9.17464652320962. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 47.12414862246475 (T) = (0 -4.890088527960731) / Math.Sqrt((0.09375351684914386 / (299)) + (0.24503147610270737 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.502286845052486 = (9.825114082983022 - 4.890088527960731) / 9.825114082983022 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.Next_long_long ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.81010681201015 < 37.28209525446735. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 148.21953225646644 (T) = (0 -25.958858441265157) / Math.Sqrt((0.5008425135491653 / (299)) + (0.15470987762870084 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.34355618066735055 = (39.54467644718678 - 25.958858441265157) / 39.54467644718678 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes_span ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.272671888473297 < 6.033332543168028. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 337.16681735067345 (T) = (0 -2285.036103313542) / Math.Sqrt((19632.457695039087 / (299)) + (1763.0959064640897 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.637716846110646 = (6307.320886389936 - 2285.036103313542) / 6307.320886389936 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextDouble ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.7223619297971013 < 7.728743487025749. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 129.7546060545115 (T) = (0 -3.81987069661909) / Math.Sqrt((0.03821574644193637 / (299)) + (0.023771299525365906 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5365200769204752 = (8.241717723690199 - 3.81987069661909) / 8.241717723690199 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.Next_long ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.709588785096553 < 33.30541520378076. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 48.0822609242298 (T) = (0 -22.966630789474106) / Math.Sqrt((0.8879327050486628 / (299)) + (1.365393806538549 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.34806423062963626 = (35.228364309041 - 22.966630789474106) / 35.228364309041 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(String).ConcurrentDictionary(Size%3a%20512).html>) 14.61 μs 8.51 μs 0.58 0.42 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.514796099122332 < 10.619822463451227. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 36.178262735261534 (T) = (0 -8747.36742086458) / Math.Sqrt((4347335.902477132 / (299)) + (53489.03142371484 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.35014047888695693 = (13460.397419249284 - 8747.36742086458) / 13460.397419249284 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TwoArguments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments.html>) 65.77 ns 48.00 ns 0.73 0.18 False
[TwoArguments_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_FilteredByLevel.html>) 56.52 ns 41.76 ns 0.74 0.13 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 47.99747214401499 < 59.23424110379324. IsChangePoint: Marked as a change because one of 1/8/2023 4:08:48 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 48.734588596243604 (T) = (0 -48.12300736430829) / Math.Sqrt((6.268258623548179 / (299)) + (1.3647323814822934 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.2258587450063804 = (62.16308335706113 - 48.12300736430829) / 62.16308335706113 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_FilteredByLevel ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 41.75645503212639 < 52.25469571824994. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 65.15696646376396 (T) = (0 -41.02728894517082) / Math.Sqrt((6.196753717337477 / (299)) + (0.7325964381985799 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.26961150061625333 = (56.17187151740056 - 41.02728894517082) / 56.17187151740056 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).ImmutableHashSet(Size%3a%20512).html>) 105.21 μs 91.43 μs 0.87 0.00 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).ConcurrentDictionary(Size%3a%20512).html>) 52.13 μs 22.45 μs 0.43 0.00 False
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).ImmutableDictionary(Size%3a%20512).html>) 112.76 μs 106.01 μs 0.94 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 91.43372560399158 < 100.03913040429939. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 33.30386358493316 (T) = (0 -94289.66020799885) / Math.Sqrt((1336172.5382646585 / (299)) + (1885480.3557159705 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09589308506187237 = (104290.38717666654 - 94289.66020799885) / 104290.38717666654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.448883986216444 < 49.52603838491275. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 842.1429643078724 (T) = (0 -22258.687527802536) / Math.Sqrt((125248.22187510475 / (299)) + (18766.729838852865 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.5743459403527975 = (52292.90552579562 - 22258.687527802536) / 52292.90552579562 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 106.0124426627794 < 107.9049582316472. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 29.692397203702168 (T) = (0 -105604.61074347876) / Math.Sqrt((2147874.172503701 / (299)) + (1499459.876215714 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07032279942591149 = (113592.7725002468 - 105604.61074347876) / 113592.7725002468 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CreateAddAndClear(String).ImmutableList(Size%3a%20512).html>) 100.26 μs 85.01 μs 0.85 0.06 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<String>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 85.00563400135869 < 92.46646082891554. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 53.332816782920034 (T) = (0 -84397.90886675654) / Math.Sqrt((3713031.12065543 / (299)) + (944941.4396825405 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.12944791119834279 = (96947.56919477726 - 84397.90886675654) / 96947.56919477726 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinqQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(IntStruct).LinqQuery(Size%3a%20512).html>) 28.86 μs 26.08 μs 0.90 0.01 False
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(IntStruct).LinqOrderByExtension(Size%3a%20512).html>) 28.84 μs 25.97 μs 0.90 0.03 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(IntStruct).Array_Comparison(Size%3a%20512).html>) 22.92 μs 18.44 μs 0.80 0.01 False
[Array_ComparerStruct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(IntStruct).Array_ComparerStruct(Size%3a%20512).html>) 26.89 μs 22.37 μs 0.83 0.05 False
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(IntStruct).Array_ComparerClass(Size%3a%20512).html>) 22.98 μs 18.44 μs 0.80 0.01 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntStruct>.LinqQuery(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 26.07798655714286 < 27.458514290109893. IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/5/2023 12:24:13 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 53.88653603898509 (T) = (0 -26015.286420499495) / Math.Sqrt((534695.6630006544 / (299)) + (45847.90994504577 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.11417772571347258 = (29368.516886136247 - 26015.286420499495) / 29368.516886136247 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.970567353846153 < 27.4239234575. IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/5/2023 12:24:13 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 71.11461898336232 (T) = (0 -25939.9209080172) / Math.Sqrt((544659.7256749384 / (299)) + (12538.467798241732 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1176776987805587 = (29399.597938492676 - 25939.9209080172) / 29399.597938492676 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.43644244 < 21.878405017416664. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 31.526207661840044 (T) = (0 -19797.819298251314) / Math.Sqrt((136313.97227744124 / (299)) + (210150.7639878256 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1349224272766101 = (22885.599999922437 - 19797.819298251314) / 22885.599999922437 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array_ComparerStruct(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.37240914666667 < 25.55342424947619. IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/10/2022 5:39:01 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 17.683002603373282 (T) = (0 -24473.252664948384) / Math.Sqrt((156947.0931784465 / (299)) + (430976.1494187926 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09295063517986349 = (26981.16950867532 - 24473.252664948384) / 26981.16950867532 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.4388665 < 21.873265384642856. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 23.19179365099016 (T) = (0 -19854.28844761905) / Math.Sqrt((134026.3141580773 / (299)) + (362355.9927143319 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1318978914200917 = (22870.913745501486 - 19854.28844761905) / 22870.913745501486 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyFalse(String%2c%20String).ConcurrentDictionary(Size%3a%20512).html>) 12.25 μs 6.66 μs 0.54 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.663023097210422 < 11.748671691706072. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 110.56400289209353 (T) = (0 -6406.890600398872) / Math.Sqrt((602101.7772353009 / (299)) + (20902.981927229146 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.4844020774990289 = (12426.137346173666 - 6406.890600398872) / 12426.137346173666 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CreateAddAndClear(Int32).ImmutableSortedSet(Size%3a%20512).html>) 147.05 μs 135.87 μs 0.92 0.01 False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CreateAddAndClear(Int32).ImmutableList(Size%3a%20512).html>) 88.33 μs 78.86 μs 0.89 0.04 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CreateAddAndClear(Int32).SortedSet(Size%3a%20512).html>) 40.08 μs 35.84 μs 0.89 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 135.87170452586207 < 137.8832635865162. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 19.628171889518477 (T) = (0 -135225.8998448027) / Math.Sqrt((3322938.817495271 / (299)) + (3255940.73156555 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.05473002950434281 = (143055.32182927203 - 135225.8998448027) / 143055.32182927203 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 78.86234631009616 < 83.68808838541162. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 41.82398464347709 (T) = (0 -76319.279384747) / Math.Sqrt((2368277.547910533 / (299)) + (1502231.1689955539 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1291498989920522 = (87637.67644559356 - 76319.279384747) / 87637.67644559356 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.840918937969924 < 37.74273932737335. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 17.524686474263074 (T) = (0 -36356.595274818166) / Math.Sqrt((323360.2824250802 / (299)) + (625530.287137814 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07498007562714717 = (39303.58073040134 - 36356.595274818166) / 39303.58073040134 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryAddGiventSize(String).ConcurrentDictionary(Count%3a%20512).html>) 41.17 μs 35.07 μs 0.85 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddGiventSize<String>.ConcurrentDictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.07373425885912 < 38.570202290017704. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 64.44427259227038 (T) = (0 -34801.57213695457) / Math.Sqrt((840395.1272868784 / (299)) + (136529.4902650983 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.14954079216249427 = (40920.91874158882 - 34801.57213695457) / 40920.91874158882 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(String).ImmutableList(Size%3a%20512).html>) 2.92 ms 2.38 ms 0.82 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.3826042869047614 < 2.922134682453125. IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 11.992583056970622 (T) = (0 -2328604.1149054626) / Math.Sqrt((143348777660.03357 / (299)) + (4541476590.342018 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1188463519252185 = (2642676.586533112 - 2328604.1149054626) / 2642676.586533112 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyTrue(String%2c%20String).ConcurrentDictionary(Size%3a%20512).html>) 15.10 μs 9.63 μs 0.64 0.13 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<String, String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.633940779077117 < 14.501075001952826. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 71.981246324902 (T) = (0 -8706.911923592754) / Math.Sqrt((773980.736826058 / (299)) + (152381.1330045945 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.4464176594356285 = (15728.305051631789 - 8706.911923592754) / 15728.305051631789 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueTrue(String%2c%20String).ConcurrentDictionary(Size%3a%20512).html>) 14.94 μs 9.14 μs 0.61 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<String, String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.135430487677528 < 14.19971246768443. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 82.81345222518705 (T) = (0 -8749.947947309334) / Math.Sqrt((796564.6044293563 / (299)) + (84850.17538912917 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.43319508708532034 = (15437.31846344538 - 8749.947947309334) / 15437.31846344538 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Int32).IndexOfValue(Size%3a%20512).html>) 19.13 ns 16.02 ns 0.84 0.44 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.IndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.022935931713324 < 18.931369774984834. IsChangePoint: Marked as a change because one of 1/3/2023 3:01:43 PM, 1/14/2023 10:43:56 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 8.721114889249796 (T) = (0 -17.788061146712803) / Math.Sqrt((1.338001482095214 / (299)) + (1.0908726239134894 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10232900910633325 = (19.815791450500242 - 17.788061146712803) / 19.815791450500242 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).ConcurrentDictionary(Size%3a%20512).html>) 3.07 μs 1.78 μs 0.58 0.18 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 29.28 μs 25.33 μs 0.87 0.00 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<Int32, Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.7811892344429192 < 2.8588196111036104. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/5/2023 12:24:13 PM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 77.19634272560914 (T) = (0 -1846.9589527815378) / Math.Sqrt((50064.77581603888 / (299)) + (2049.734356829728 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.40289180812399134 = (3093.173026112267 - 1846.9589527815378) / 3093.173026112267 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.33078208931466 < 27.797193586064477. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 71.82769513125835 (T) = (0 -25141.071717980263) / Math.Sqrt((202428.13291527607 / (299)) + (46630.497680894994 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.13125464195234005 = (28939.517759818646 - 25141.071717980263) / 28939.517759818646 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).ConcurrentDictionary(Size%3a%20512).html>) 3.00 μs 2.59 μs 0.86 0.16 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 25.17 μs 22.98 μs 0.91 0.04 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.5882191850874303 < 2.8556086117961508. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 8.835582256932375 (T) = (0 -2464.47828640434) / Math.Sqrt((9867.450974462972 / (299)) + (83745.6873189782 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.18177263672552413 = (3011.972462692654 - 2464.47828640434) / 3011.972462692654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.980784921071955 < 24.029970468930006. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/5/2023 12:24:13 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 31.91892824419397 (T) = (0 -23476.524696088887) / Math.Sqrt((296379.9703892691 / (299)) + (62574.49126735114 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.07766346020335393 = (25453.317398945204 - 23476.524696088887) / 25453.317398945204 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSet(String).ConcurrentDictionary(Size%3a%20512).html>) 27.30 μs 20.75 μs 0.76 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 20.747816498655915 < 26.014325693434166. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 76.43581620334889 (T) = (0 -21138.126257536132) / Math.Sqrt((818971.2516586663 / (299)) + (84637.64682174528 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.22687942389763346 = (27341.306014777827 - 21138.126257536132) / 27341.306014777827 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetDateTimeOffset - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetDateTimeOffset.html>) 5.06 μs 4.66 μs 0.92 0.03 False
[GetDateTime - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetDateTime.html>) 3.78 μs 3.42 μs 0.90 0.00 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.663060854576625 < 4.80302387208541. IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 29.667879174213745 (T) = (0 -4754.847782119512) / Math.Sqrt((9184.220178553198 / (299)) + (1327.9201574544084 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.056199586935866755 = (5037.980187656911 - 4754.847782119512) / 5037.980187656911 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetDateTime ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.4165796890941555 < 3.5796783462417885. IsChangePoint: Marked as a change because one of 12/13/2022 4:26:35 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 80.8513294610489 (T) = (0 -3444.9439575752845) / Math.Sqrt((1572.2314757891183 / (299)) + (158.396178206084 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07560700129576475 = (3726.7092701959264 - 3444.9439575752845) / 3726.7092701959264 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.AddGivenSize(String).ConcurrentDictionary(Size%3a%20512).html>) 41.34 μs 35.22 μs 0.85 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.22076170993228 < 38.84054886396097. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 70.90337697668669 (T) = (0 -34749.51870828038) / Math.Sqrt((854179.7029966972 / (299)) + (101240.78944374315 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.14981680604479158 = (40872.977677457064 - 34749.51870828038) / 40872.977677457064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 29.30 μs 25.76 μs 0.88 0.01 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ConcurrentDictionary(Size%3a%20512).html>) 2.61 μs 1.91 μs 0.73 0.18 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.75832739830186 < 27.85262015580603. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 30.287754449568826 (T) = (0 -25406.877074914475) / Math.Sqrt((115678.35200927389 / (299)) + (330754.04198856547 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.12894897376032596 = (29168.06973363653 - 25406.877074914475) / 29168.06973363653 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.9091404216749985 < 2.4398538728026895. IsChangePoint: Marked as a change because one of 12/7/2022 8:33:47 PM, 12/10/2022 11:25:25 PM, 1/5/2023 11:35:29 PM, 1/9/2023 12:42:28 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 35.24623868246446 (T) = (0 -1883.3834835015361) / Math.Sqrt((44714.71438595447 / (299)) + (4374.889628364179 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.2588830773380486 = (2541.27712633626 - 1883.3834835015361) / 2541.27712633626 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BenchStringSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/ByteMark.BenchStringSort.html>) 1.30 secs 1.21 secs 0.92 0.00 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### ByteMark.BenchStringSort ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.2062822757142857 < 1.2375962653342947. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 105.94479718649771 (T) = (0 -1204637727.0348155) / Math.Sqrt((67822939120031.49 / (299)) + (11697834585722.697 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.07114796500018511 = (1296910252.2718332 - 1204637727.0348155) / 1296910252.2718332 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueTrue(Int32%2c%20Int32).Dictionary(Size%3a%20512).html>) 3.93 μs 3.71 μs 0.94 0.16 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 25.88 μs 23.37 μs 0.90 0.02 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.712179201824172 < 3.7356214170716284. IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/10/2022 11:25:25 PM, 1/3/2023 6:29:42 AM, 1/5/2023 12:24:13 PM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 9.414113505717578 (T) = (0 -3246.589227609837) / Math.Sqrt((116409.97939700121 / (299)) + (43102.06785569811 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.12320962021195417 = (3702.811187771772 - 3246.589227609837) / 3702.811187771772 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 23.368468616538543 < 24.60491403418136. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 18.001833501114493 (T) = (0 -23846.679094593) / Math.Sqrt((123171.47309832333 / (299)) + (297531.0667884457 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08021991613512881 = (25926.50081570631 - 23846.679094593) / 25926.50081570631 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryGetValueFalse(String%2c%20String).ConcurrentDictionary(Size%3a%20512).html>) 12.23 μs 6.63 μs 0.54 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<String, String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.628339366723021 < 11.604206950787587. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 112.25021032291026 (T) = (0 -6380.614820499255) / Math.Sqrt((617258.3971152643 / (299)) + (14392.259005403263 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.47842524586121493 = (12233.365917096222 - 6380.614820499255) / 12233.365917096222 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(Int32).ConcurrentDictionary(Size%3a%20512).html>) 9.56 μs 5.06 μs 0.53 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.055511664657021 < 9.065475291741496. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 482.75187306157596 (T) = (0 -5042.650608548572) / Math.Sqrt((3163.7559096305263 / (299)) + (1578.7578610473815 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.46993441948007164 = (9513.257970084303 - 5042.650608548572) / 9513.257970084303 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int64.ToString(value%3a%20-9223372036854775808).html>) 31.88 ns 28.43 ns 0.89 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.ToString(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 28.43121847752731 < 30.203475606011278. IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM. IsImprovementStdDev: Marked as improvement because 14.706676867951513 (T) = (0 -26.8179906824437) / Math.Sqrt((2.7755642636134223 / (299)) + (1.2447447961940379 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.12337323069732174 = (30.59225615911358 - 26.8179906824437) / 30.59225615911358 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)