search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.97k stars 4.65k forks source link

[Mono][Perf][Wasm] Regression across many collection related benchmarks #61624

Closed naricc closed 2 years ago

naricc commented 2 years ago

Based on the time frame, it seems likely to have been caused by one of these two changes: https://github.com/dotnet/runtime/compare/9e795c014b0be513c84f96427c544bae486bb101...e34c3953f8d69b06233c576bf2c2818d0ea0ebc3 @vargaz

Automated performance issue below.

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).SortedDictionary(Size%3a%20512).html>) 456.60 μs 671.74 μs 1.47 0.62 False
[IEnumerable - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).IEnumerable(Size%3a%20512).html>) 77.45 μs 595.24 μs 7.69 0.60 True

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) ```log ``` #### System.Collections.IterateForEach<Int32>.IEnumerable(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20IList).html>) 68.41 μs 223.32 μs 3.26 0.60 True
[OrderBy - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.OrderBy(input%3a%20IEnumerable).html>) 273.59 μs 543.24 μs 1.99 0.50 False
[Concat_Once - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Concat_Once(input%3a%20IEnumerable).html>) 69.13 μs 295.30 μs 4.27 0.55 True
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20IEnumerable).html>) 22.74 μs 122.35 μs 5.38 0.59 True
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20List).html>) 25.20 μs 131.03 μs 5.20 0.58 True
[Select - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20IList).html>) 37.87 μs 210.83 μs 5.57 0.52 True
[LastWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input%3a%20IList).html>) 858.35 ns 1.90 μs 2.21 0.47 True
[WhereAny_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input%3a%20IEnumerable).html>) 24.89 μs 138.60 μs 5.57 0.58 True
[Sum - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Sum(input%3a%20IEnumerable).html>) 16.43 μs 117.65 μs 7.16 0.55 True
[Where - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20Array).html>) 33.01 μs 94.80 μs 2.87 0.60 True
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20List).html>) 17.74 μs 70.00 μs 3.95 0.62 True
[ToDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20List).html>) 48.24 μs 191.04 μs 3.96 0.54 True
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Array).html>) 46.46 μs 101.97 μs 2.19 0.60 False
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20List).html>) 31.91 μs 123.09 μs 3.86 0.53 True
[AppendPrepend - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.AppendPrepend(input%3a%20IEnumerable).html>) 147.77 μs 401.27 μs 2.72 0.57 False
[ToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.ToArray(input%3a%20IEnumerable).html>) 44.56 μs 157.27 μs 3.53 0.51 True
[OrderByDescending - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.OrderByDescending(input%3a%20IEnumerable).html>) 370.09 μs 616.25 μs 1.67 0.54 False
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20IEnumerable).html>) 21.07 μs 117.10 μs 5.56 0.56 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList) ```log ``` #### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Select(input: IList) ```log ``` #### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList) ```log ``` #### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Where(input: Array) ```log ``` #### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) ```log ``` #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Bag - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Concurrent.IsEmpty(String).Bag(Size%3a%20512).html>) 516.31 ns 1.67 μs 3.23 0.48 True
[Bag - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Concurrent.IsEmpty(String).Bag(Size%3a%200).html>) 510.51 ns 1.62 μs 3.17 0.51 True

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.IsEmpty<String>.Bag(Size: 512) ```log ``` #### System.Collections.Concurrent.IsEmpty<String>.Bag(Size: 0) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrueComparer(Int32).SortedSet(Size%3a%20512).html>) 2.37 ms 6.24 ms 2.64 0.44 True

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).Stack(Size%3a%20512).html>) 109.56 μs 547.95 μs 5.00 0.65 True
[SortedList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).SortedList(Size%3a%20512).html>) 178.04 μs 657.43 μs 3.69 0.57 True

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.Stack(Size: 512) ```log ``` #### System.Collections.IterateForEach<String>.SortedList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.TryAddDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.TryAddDefaultSize(String).ConcurrentDictionary(Count%3a%20512).html>) 2.03 ms 2.45 ms 1.21 0.47 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddDefaultSize<String>.ConcurrentDictionary(Count: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.AddGivenSize(Int32).SortedList(Size%3a%20512).html>) 2.65 ms 4.60 ms 1.74 0.47 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.SortedList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).Array(Size%3a%20512).html>) 1.00 ms 1.73 ms 1.73 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<BigStruct>.Array(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinqQuery - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).LinqQuery(Size%3a%20512).html>) 838.70 μs 1.60 ms 1.91 0.07 True
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).LinqOrderByExtension(Size%3a%20512).html>) 846.71 μs 1.57 ms 1.86 0.05 True

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<Int32>.LinqQuery(Size: 512) ```log ``` #### System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%2012345).html>) 4.85 μs 6.64 μs 1.37 0.40 False
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%201.7976931348623157E%2b308).html>) 5.75 μs 7.01 μs 1.22 0.43 False
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%20-1.7976931348623157E%2b308).html>) 5.55 μs 7.11 μs 1.28 0.38 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345) ```log ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308) ```log ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Concurrent.IsEmpty(Int32).Dictionary(Size%3a%20512).html>) 356.17 ns 1.01 μs 2.83 0.54 True

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(IndexViewModel).JsonNet.html>) 1.85 ms 2.09 ms 1.13 0.37 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_ ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).ConcurrentDictionary(Size%3a%20512).html>) 1.53 ms 2.05 ms 1.34 0.54 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).ConcurrentQueue(Size%3a%20512).html>) 238.71 μs 952.23 μs 3.99 0.57 True

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.ConcurrentDictionary(Size: 512) ```log ``` #### System.Collections.CtorFromCollection<String>.ConcurrentQueue(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.TryAddDefaultSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.TryAddDefaultSize(Int32).ConcurrentDictionary(Count%3a%20512).html>) 1.46 ms 2.02 ms 1.38 0.48 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddDefaultSize<Int32>.ConcurrentDictionary(Count: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.Concurrent.Count(String).Queue(Size%3a%20512).html>) 490.34 ns 1.11 μs 2.27 0.40 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<String>.Queue(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(Int32).ConcurrentStack(Size%3a%20512).html>) 137.63 μs 659.18 μs 4.79 0.61 True
[ImmutableList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(Int32).ImmutableList(Size%3a%20512).html>) 425.63 μs 625.45 μs 1.47 0.45 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<Int32>.ConcurrentStack(Size: 512) ```log ``` #### System.Collections.CtorFromCollection<Int32>.ImmutableList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6cf529168a8dcdfb158738d46be40b1867fd1bfa
Compare c1d6d849820957918c69f2d1c1801cf3cb2578c9
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Concat_CharEnumerable - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_String.Concat_CharEnumerable.html>) 1.01 ms 2.07 ms 2.04 0.50 True

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Concat_CharEnumerable ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
vargaz commented 2 years ago

The second change did change behavior by allowing more interpreter->aot transitions. In some cases, that might improve performance, in some cases it might decrease it. Not sure what to do here.

ghost commented 2 years ago

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

Issue Details
Based on the time frame, it seems likely to have been caused by one of these two changes: https://github.com/dotnet/runtime/compare/9e795c014b0be513c84f96427c544bae486bb101...e34c3953f8d69b06233c576bf2c2818d0ea0ebc3 @vargaz Automated performance issue below. ### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.IterateForEach<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [SortedDictionary - Duration of single invocation]() | 456.60 μs | 671.74 μs | 1.47 | 0.62 | False | | | [IEnumerable - Duration of single invocation]() | 77.45 μs | 595.24 μs | 7.69 | 0.60 | True | | | ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) ```log ``` #### System.Collections.IterateForEach<Int32>.IEnumerable(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Linq.Tests.Perf_Enumerable Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [SelectToArray - Duration of single invocation]() | 68.41 μs | 223.32 μs | 3.26 | 0.60 | True | | | [OrderBy - Duration of single invocation]() | 273.59 μs | 543.24 μs | 1.99 | 0.50 | False | | | [Concat_Once - Duration of single invocation]() | 69.13 μs | 295.30 μs | 4.27 | 0.55 | True | | | [SingleWithPredicate_FirstElementMatches - Duration of single invocation]() | 22.74 μs | 122.35 μs | 5.38 | 0.59 | True | | | [SingleWithPredicate_FirstElementMatches - Duration of single invocation]() | 25.20 μs | 131.03 μs | 5.20 | 0.58 | True | | | [Select - Duration of single invocation]() | 37.87 μs | 210.83 μs | 5.57 | 0.52 | True | | | [LastWithPredicate_FirstElementMatches - Duration of single invocation]() | 858.35 ns | 1.90 μs | 2.21 | 0.47 | True | | | [WhereAny_LastElementMatches - Duration of single invocation]() | 24.89 μs | 138.60 μs | 5.57 | 0.58 | True | | | [Sum - Duration of single invocation]() | 16.43 μs | 117.65 μs | 7.16 | 0.55 | True | | | [Where - Duration of single invocation]() | 33.01 μs | 94.80 μs | 2.87 | 0.60 | True | | | [WhereSingleOrDefault_LastElementMatches - Duration of single invocation]() | 17.74 μs | 70.00 μs | 3.95 | 0.62 | True | | | [ToDictionary - Duration of single invocation]() | 48.24 μs | 191.04 μs | 3.96 | 0.54 | True | | | [SelectToArray - Duration of single invocation]() | 46.46 μs | 101.97 μs | 2.19 | 0.60 | False | | | [AnyWithPredicate_LastElementMatches - Duration of single invocation]() | 31.91 μs | 123.09 μs | 3.86 | 0.53 | True | | | [AppendPrepend - Duration of single invocation]() | 147.77 μs | 401.27 μs | 2.72 | 0.57 | False | | | [ToArray - Duration of single invocation]() | 44.56 μs | 157.27 μs | 3.53 | 0.51 | True | | | [OrderByDescending - Duration of single invocation]() | 370.09 μs | 616.25 μs | 1.67 | 0.54 | False | | | [AnyWithPredicate_LastElementMatches - Duration of single invocation]() | 21.07 μs | 117.10 μs | 5.56 | 0.56 | True | | | ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList) ```log ``` #### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Select(input: IList) ```log ``` #### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList) ```log ``` #### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.Where(input: Array) ```log ``` #### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) ```log ``` #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List) ```log ``` #### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable) ```log ``` #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.Concurrent.IsEmpty<String> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Bag - Duration of single invocation]() | 516.31 ns | 1.67 μs | 3.23 | 0.48 | True | | | [Bag - Duration of single invocation]() | 510.51 ns | 1.62 μs | 3.17 | 0.51 | True | | | ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty<String>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.IsEmpty<String>.Bag(Size: 512) ```log ``` #### System.Collections.Concurrent.IsEmpty<String>.Bag(Size: 0) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.ContainsTrueComparer<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [SortedSet - Duration of single invocation]() | 2.37 ms | 6.24 ms | 2.64 | 0.44 | True | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.IterateForEach<String> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Stack - Duration of single invocation]() | 109.56 μs | 547.95 μs | 5.00 | 0.65 | True | | | [SortedList - Duration of single invocation]() | 178.04 μs | 657.43 μs | 3.69 | 0.57 | True | | | ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<String>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.Stack(Size: 512) ```log ``` #### System.Collections.IterateForEach<String>.SortedList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.TryAddDefaultSize<String> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [ConcurrentDictionary - Duration of single invocation]() | 2.03 ms | 2.45 ms | 1.21 | 0.47 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize<String>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddDefaultSize<String>.ConcurrentDictionary(Count: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.AddGivenSize<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [SortedList - Duration of single invocation]() | 2.65 ms | 4.60 ms | 1.74 | 0.47 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.SortedList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.Sort<BigStruct> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Array - Duration of single invocation]() | 1.00 ms | 1.73 ms | 1.73 | 0.03 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort<BigStruct>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<BigStruct>.Array(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.Sort<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [LinqQuery - Duration of single invocation]() | 838.70 μs | 1.60 ms | 1.91 | 0.07 | True | | | [LinqOrderByExtension - Duration of single invocation]() | 846.71 μs | 1.57 ms | 1.86 | 0.05 | True | | | ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<Int32>.LinqQuery(Size: 512) ```log ``` #### System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Buffers.Text.Tests.Utf8FormatterTests Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [FormatterDouble - Duration of single invocation]() | 4.85 μs | 6.64 μs | 1.37 | 0.40 | False | | | [FormatterDouble - Duration of single invocation]() | 5.75 μs | 7.01 μs | 1.22 | 0.43 | False | | | [FormatterDouble - Duration of single invocation]() | 5.55 μs | 7.11 μs | 1.28 | 0.38 | False | | | ![graph]() ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345) ```log ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308) ```log ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.Concurrent.IsEmpty<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Dictionary - Duration of single invocation]() | 356.17 ns | 1.01 μs | 2.83 | 0.54 | True | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [JsonNet_ - Duration of single invocation]() | 1.85 ms | 2.09 ms | 1.13 | 0.37 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_ ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.CtorFromCollection<String> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [ConcurrentDictionary - Duration of single invocation]() | 1.53 ms | 2.05 ms | 1.34 | 0.54 | False | | | [ConcurrentQueue - Duration of single invocation]() | 238.71 μs | 952.23 μs | 3.99 | 0.57 | True | | | ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection<String>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.ConcurrentDictionary(Size: 512) ```log ``` #### System.Collections.CtorFromCollection<String>.ConcurrentQueue(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.TryAddDefaultSize<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [ConcurrentDictionary - Duration of single invocation]() | 1.46 ms | 2.02 ms | 1.38 | 0.48 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddDefaultSize<Int32>.ConcurrentDictionary(Count: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.Concurrent.Count<String> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Queue - Duration of single invocation]() | 490.34 ns | 1.11 μs | 2.27 | 0.40 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count<String>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<String>.Queue(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Collections.CtorFromCollection<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [ConcurrentStack - Duration of single invocation]() | 137.63 μs | 659.18 μs | 4.79 | 0.61 | True | | | [ImmutableList - Duration of single invocation]() | 425.63 μs | 625.45 μs | 1.47 | 0.45 | False | | | ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<Int32>.ConcurrentStack(Size: 512) ```log ``` #### System.Collections.CtorFromCollection<Int32>.ImmutableList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [6cf529168a8dcdfb158738d46be40b1867fd1bfa](https://github.com/dotnet/runtime/commit/6cf529168a8dcdfb158738d46be40b1867fd1bfa) Compare | [c1d6d849820957918c69f2d1c1801cf3cb2578c9](https://github.com/dotnet/runtime/commit/c1d6d849820957918c69f2d1c1801cf3cb2578c9) Diff | [Diff](https://github.com/dotnet/runtime/compare/6cf529168a8dcdfb158738d46be40b1867fd1bfa...c1d6d849820957918c69f2d1c1801cf3cb2578c9) ### Regressions in System.Tests.Perf_String Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Concat_CharEnumerable - Duration of single invocation]() | 1.01 ms | 2.07 ms | 2.04 | 0.50 | True | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Concat_CharEnumerable ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Author: naricc
Assignees: -
Labels: `arch-wasm`, `tenet-performance`, `untriaged`, `area-Codegen-AOT-mono`, `runtime-mono`
Milestone: -
naricc commented 2 years ago

@vargaz If you say this is a worthwhile tradeoff, I can just close the issue. So long as its documented somewhere.

What do you think @SamMonoRT?

SamMonoRT commented 2 years ago

@naricc - What was the % improvement in other benchmarks, both interpreter-wasm and aot-wasm ones to justify the tradeoff ?

vargaz commented 2 years ago

Would it be possible to rerun the benchmarks with latest master ?

naricc commented 2 years ago

@vargaz Yes, I can do that. They are actually run against the lastest master a few times a day on a regular cadence, so I can just look.

@SamMonoRT Let me dig into that; I should be able to search for improvement issues filed around this time to find out.

radical commented 2 years ago

Is this planned for 7.0?

SamMonoRT commented 2 years ago

I believe all those have improved. We can close this issue. cc @naricc

naricc commented 2 years ago

Looking at the perf data, it seems like this specific spike has been resolved. There does seem to be some kind of other creep up word in these benchmarks that we might want to look at in the future, example: https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_ubuntu%2018.04_CompliationMode%3dwasm_RunKind%3dmicro%2fSystem.Collections.AddGivenSize(Int32).Dictionary(Size%3a%20512).html

Something happened between January and March where we are missing data. But yeah, I think it is safe to close this.

radical commented 2 years ago

Jan-Mar was when the perf builds were broken.