[Perf] Linux/x64: 9 Regressions on 10/19/2023 6:52:58 PM

performanceautofiler[bot] commented 11 months ago

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[AllocateUninitializedArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.Tests.Perf_GC(Byte).AllocateUninitializedArray(length%3a%2010000%2c%20pinned%3a%20False).html>) 📈 - ADX Test Multi Config Graph	145.34 ns	154.83 ns	1.07	0.02	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'

### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: False) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.Collections.CtorDefaultSize(Int32).Dictionary.html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.69 ns	8.79 ns	1.14	0.09	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;Int32&gt;*'

### Payloads [Baseline]() [Compare]() ### System.Collections.CtorDefaultSize<Int32>.Dictionary #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.StackWalk

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[Walk - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/PerfLabTests.StackWalk.Walk.html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	45.45 ms	51.55 ms	1.13	0.03	True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.StackWalk*'

### Payloads [Baseline]() [Compare]() ### PerfLabTests.StackWalk.Walk #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[Exp - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.MathBenchmarks.Double.Exp.html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.49 μs	26.27 μs	1.07	0.09	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'

### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Double.Exp #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.SomeValueType%3f)).html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	111.37 ns	117.43 ns	1.05	0.17	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'

### Payloads [Baseline]() [Compare]() ### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?)) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Environment

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SetEnvironmentVariable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.Tests.Perf_Environment.SetEnvironmentVariable.html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	475.22 ns	524.80 ns	1.10	0.23	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'

### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Environment.SetEnvironmentVariable #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[ctor_capacity - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.Text.Tests.Perf_StringBuilder.ctor_capacity(length%3a%20100000).html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	14.36 μs	15.81 μs	1.10	0.10	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

### Payloads [Baseline]() [Compare]() ### System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.Collections.CreateAddAndClear(Int32).ConcurrentStack(Size%3a%20512).html>) 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.86 μs	7.73 μs	1.32	0.12	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<Int32>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	04ede0dadfbf2f1082d3bd218cea6fe46bbbed4a
Compare	5f653ddd5d9bc0544fbf3ff712244e3f5a398a9f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[AllocateUninitializedArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04/System.Tests.Perf_GC(Char).AllocateUninitializedArray(length%3a%2010000%2c%20pinned%3a%20False).html>) 📈 - ADX Test Multi Config Graph	217.78 ns	239.15 ns	1.10	0.01	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'

### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: False) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

cincuranet commented 11 months ago

Diff: https://github.com/dotnet/runtime/compare/de9aef390b1ccc736b3a9183645aa561a34f7f19...bb3597aeaaeb87920b848e71a7b152a202e672bd. Seems like https://github.com/dotnet/runtime/pull/93704 again. cc @AndyAyersMS

AndyAyersMS commented 11 months ago

Do we know why this has the PGO label?

Some of these tests likely spend time in native code in the runtime, so it's possible the native PGO is out of date or something.

let's look again and see if it fixes itself.

cincuranet commented 11 months ago

PGO wasn't updated. The code we have for that is not 100% fool proof and failed here (I'll check what I can do to make it more reliable).

dotnet / perf-autofiling-issues