search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 6/29/2022 1:11:29 AM #6568

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(Int32).SortedList(Size%3a%20512).html>) 845.41 μs 374.43 μs 0.44 0.06 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(Int32).ImmutableSortedSet(Size%3a%20512).html>) 956.19 μs 610.80 μs 0.64 0.02 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(Int32).SortedSet(Size%3a%20512).html>) 793.25 μs 429.70 μs 0.54 0.02 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 374.43095089285714 < 810.1842635044642. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 227.08158743857135 (T) = (0 -376120.8686908124) / Math.Sqrt((1280438237.2741232 / (299)) + (8580211.19410225 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.5619362240605862 = (858598.4264145858 - 376120.8686908124) / 858598.4264145858 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 610.7988786858974 < 913.6935857438289. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 135.71477501611545 (T) = (0 -603033.4803686213) / Math.Sqrt((1593485354.912962 / (299)) + (67710030.94527641 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.3758488837878269 = (966165.8286029996 - 603033.4803686213) / 966165.8286029996 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 429.69618665540537 < 750.4459532820513. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 165.9864520668582 (T) = (0 -433712.57497148484) / Math.Sqrt((1061502853.430323 / (299)) + (41811206.50282233 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4529386046720944 = (792804.2056623643 - 433712.57497148484) / 792804.2056623643 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).Array(Size%3a%20512).html>) 1.12 ms 666.64 μs 0.59 0.01 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).List(Size%3a%20512).html>) 1.12 ms 655.80 μs 0.59 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<BigStruct>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 666.6414571857144 < 1.061881343935357. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 274.7782848669194 (T) = (0 -659457.7347074468) / Math.Sqrt((627177062.3770927 / (299)) + (27283344.202417687 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41230670562078797 = (1122112.0625581418 - 659457.7347074468) / 1122112.0625581418 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<BigStruct>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 655.7970333666666 < 1.0672148167317617. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 232.7393370789237 (T) = (0 -661195.0490530015) / Math.Sqrt((637670761.042802 / (299)) + (66427849.71793315 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4109447161896637 = (1122466.8842218427 - 661195.0490530015) / 1122466.8842218427 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 654.66 μs 384.02 μs 0.59 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 384.0223575203252 < 621.7033102063301. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 175.3957116564391 (T) = (0 -387950.97146240016) / Math.Sqrt((467343209.99102044 / (299)) + (30141323.855679393 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4107712965700513 = (658404.7403055989 - 387950.97146240016) / 658404.7403055989 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.AddGivenSize(Int32).SortedList(Size%3a%20512).html>) 943.30 μs 673.04 μs 0.71 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 673.0416666666666 < 892.3594314403884. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 29.145301352150714 (T) = (0 -680072.7497735185) / Math.Sqrt((24361500984.500095 / (299)) + (65468240.695813194 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.2811011689924607 = (945992.2877053425 - 680072.7497735185) / 945992.2877053425 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).Array(Size%3a%20512).html>) 685.61 μs 322.94 μs 0.47 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 322.9357142428572 < 641.8007697805493. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 182.0415357038296 (T) = (0 -324233.5571360866) / Math.Sqrt((832855727.0093045 / (299)) + (38559100.3900093 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.5234554881781033 = (680384.6211479723 - 324233.5571360866) / 680384.6211479723 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<IntClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).Array(Size%3a%20512).html>) 398.94 μs 373.23 μs 0.94 0.04 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).List(Size%3a%20512).html>) 399.47 μs 369.82 μs 0.93 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntClass&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntClass>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 373.2328266800001 < 379.3071966476667. IsChangePoint: Marked as a change because one of 5/25/2022 8:13:38 AM, 6/5/2022 6:02:14 PM, 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 20.72484217319039 (T) = (0 -368863.2388255292) / Math.Sqrt((732744999.9924022 / (299)) + (24780962.653483585 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.09028902754131651 = (405473.0018575014 - 368863.2388255292) / 405473.0018575014 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 369.82161425714287 < 378.4546350633333. IsChangePoint: Marked as a change because one of 5/25/2022 8:13:38 AM, 6/5/2022 6:02:14 PM, 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 16.381676553198403 (T) = (0 -371492.2990084542) / Math.Sqrt((682572693.6911192 / (299)) + (65103562.20679423 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.08236340466776616 = (404835.96763482824 - 371492.2990084542) / 404835.96763482824 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).List(Size%3a%20512).html>) 997.50 μs 588.41 μs 0.59 0.02 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).Array(Size%3a%20512).html>) 984.80 μs 582.39 μs 0.59 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntStruct>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 588.4084614615384 < 941.9803140019762. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 243.59824595015624 (T) = (0 -589184.3876718277) / Math.Sqrt((635894048.9336479 / (299)) + (18484522.24153326 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4012096354160041 = (983957.6962484326 - 589184.3876718277) / 983957.6962484326 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 582.3940666666666 < 935.3735783396668. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 191.66902646780488 (T) = (0 -591887.1772980021) / Math.Sqrt((529495384.56694376 / (299)) + (91342581.68136826 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.40003703359212023 = (986539.5206670349 - 591887.1772980021) / 986539.5206670349 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueTrue(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 660.68 μs 394.71 μs 0.60 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 394.70733125000004 < 627.3106090908788. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 194.8357346817769 (T) = (0 -389700.61163508013) / Math.Sqrt((532925110.9941157 / (299)) + (7362136.767769038 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41305834638678834 = (663951.1938470952 - 389700.61163508013) / 663951.1938470952 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).SortedList(Size%3a%20512).html>) 947.64 μs 685.11 μs 0.72 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 685.114518439441 < 900.9852560308259. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 129.93865396328425 (T) = (0 -687846.9860471555) / Math.Sqrt((1050122362.1761982 / (299)) + (17424229.97428867 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.27378931563510095 = (947172.7707348533 - 687846.9860471555) / 947172.7707348533 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 720.75 μs 435.91 μs 0.60 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 435.9088540426587 < 686.1850001799243. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 139.17758297777064 (T) = (0 -425487.9812795426) / Math.Sqrt((698721821.3739876 / (299)) + (92196352.95938697 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41819148263638456 = (731319.615614399 - 425487.9812795426) / 731319.615614399 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).SortedList(Size%3a%20512).html>) 1.81 ms 1.33 ms 0.73 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndRemove<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.3293322930555556 < 1.721497779079861. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 118.77278547398564 (T) = (0 -1331814.0824537796) / Math.Sqrt((3766170384.820892 / (299)) + (141576701.52218297 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.2654629142709721 = (1813133.888443433 - 1331814.0824537796) / 1813133.888443433 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueFalse(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 751.41 μs 430.99 μs 0.57 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 430.9919870459402 < 692.1729748965877. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 177.05874027408177 (T) = (0 -428380.14577216416) / Math.Sqrt((609444895.4533415 / (299)) + (35244060.00736095 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41684082315194076 = (734585.277535258 - 428380.14577216416) / 734585.277535258 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(Int32).SortedList(Size%3a%20512).html>) 636.52 μs 370.98 μs 0.58 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 370.97969759778914 < 608.7793472980767. IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM. IsImprovementStdDev: Marked as improvement because 165.965178973048 (T) = (0 -372811.7455147112) / Math.Sqrt((528753253.0045457 / (299)) + (33042514.57579639 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.42070969466432934 = (643566.3467536969 - 372811.7455147112) / 643566.3467536969 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 2 years ago

@vargaz this looks like it was https://github.com/dotnet/runtime/pull/71325 + https://github.com/dotnet/runtime/pull/71349

nice win!

cc @marek-safar @SamMonoRT