search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.95k stars 4.65k forks source link

[Perf] Windows/x64: 17 Regressions on 8/16/2024 4:29:39 PM #106707

Closed performanceautofiler[bot] closed 2 weeks ago

performanceautofiler[bot] commented 3 weeks ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 2691fc8374c89bf79a366bdf62ded4cd856d357c
Compare 1a305e63ecf47e81464ca22b6ddc86d2d31c6202
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.77 ns 16.95 ns 1.57 0.24 False
10.32 ns 15.74 ns 1.52 0.19 False
10.80 ns 14.66 ns 1.36 0.27 False
11.98 ns 14.61 ns 1.22 0.24 False
10.75 ns 16.40 ns 1.52 0.26 False
27.74 ns 38.37 ns 1.38 0.42 False
10.75 ns 16.58 ns 1.54 0.27 False
10.32 ns 11.52 ns 1.12 0.18 False
10.41 ns 16.80 ns 1.61 0.24 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: Control) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Saturday, format: "D") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: UppercaseLetter) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Small(value: AllDirectories) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: Format) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Small(value: -1) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: OtherNotAssigned) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Tuesday, format: "d") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Small(value: TopDirectoryOnly) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 2691fc8374c89bf79a366bdf62ded4cd856d357c
Compare 1a305e63ecf47e81464ca22b6ddc86d2d31c6202
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
39.14 ns 49.39 ns 1.26 0.19 False
16.74 ns 37.15 ns 2.22 0.53 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Microsoft.Extensions.DependencyInjection.GetService.Scoped #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 2691fc8374c89bf79a366bdf62ded4cd856d357c
Compare 1a305e63ecf47e81464ca22b6ddc86d2d31c6202
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.11 ns 14.26 ns 1.18 0.22 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_5Injected #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 2691fc8374c89bf79a366bdf62ded4cd856d357c
Compare 1a305e63ecf47e81464ca22b6ddc86d2d31c6202
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.56 ns 20.40 ns 1.16 0.56 False
46.11 ns 70.19 ns 1.52 0.41 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'
### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Singleton #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 2691fc8374c89bf79a366bdf62ded4cd856d357c
Compare 1a305e63ecf47e81464ca22b6ddc86d2d31c6202
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
57.15 ns 84.80 ns 1.48 0.48 False
62.86 ns 118.96 ns 1.89 0.49 False
58.38 ns 70.14 ns 1.20 0.46 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'
### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassIDerived)) #### ETL Files #### Histogram #### JIT Disasms ### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string)) #### ETL Files #### Histogram #### JIT Disasms ### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassWithNoConverter)) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
dotnet-policy-service[bot] commented 3 weeks ago

Tagging subscribers to this area: @dotnet/area-system-runtime See info in area-owners.md if you want to be subscribed.

tannergooding commented 2 weeks ago

The commit range in question really only has bug fixes. There's nothing obvious that causes these regressions and they are all on the order of 4-40ns.

@AaronRobinsonMSFT, could these be caused by https://github.com/dotnet/runtime/pull/106497, that looks like the most obvious one in the commit range given what the tests are doing (often touching GetType()).

AaronRobinsonMSFT commented 2 weeks ago

Yes, it is entirely possible. There was a minor accepted performance regression in that PR. The benefit is the removal of Helper Method Frames and was thus permitted. Given the very small impact here, nanoseconds, this seems acceptable.

tannergooding commented 2 weeks ago

Closing as by design.