search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.54k stars 4.54k forks source link

[Perf] Windows/arm64: 7 Regressions on 12/12/2023 12:38:01 AM #96030

Open performanceautofiler[bot] opened 6 months ago

performanceautofiler[bot] commented 6 months ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 1473deaa50785b956edd7d078e68c0581c1b4d95
Compare 0a940f90ebc8465791870799dd496d1a524a5381
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in SeekUnroll

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.27 secs 1.43 secs 1.13 0.00 False
1.60 secs 1.75 secs 1.10 0.00 False
1.28 secs 1.43 secs 1.12 0.00 False
1.27 secs 1.43 secs 1.12 0.00 False
1.27 secs 1.43 secs 1.12 0.00 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SeekUnroll*'
### Payloads [Baseline]() [Compare]() ### SeekUnroll.Test(boxedIndex: 19) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 11) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 27) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 3) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 1) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 1473deaa50785b956edd7d078e68c0581c1b4d95
Compare 0a940f90ebc8465791870799dd496d1a524a5381
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
62.96 μs 67.24 μs 1.07 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'
### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Double.AtanPi #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 1473deaa50785b956edd7d078e68c0581c1b4d95
Compare 0a940f90ebc8465791870799dd496d1a524a5381
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
52.52 μs 57.51 μs 1.10 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Single.AtanPi #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
ghost commented 6 months ago

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch See info in area-owners.md if you want to be subscribed.

Issue Details
### Run Information Name | Value -- | -- Architecture | arm64 OS | Windows 10.0.19041 Queue | SurfaceWindows Baseline | [1473deaa50785b956edd7d078e68c0581c1b4d95](https://github.com/dotnet/runtime/commit/1473deaa50785b956edd7d078e68c0581c1b4d95) Compare | [0a940f90ebc8465791870799dd496d1a524a5381](https://github.com/dotnet/runtime/commit/0a940f90ebc8465791870799dd496d1a524a5381) Diff | [Diff](https://github.com/dotnet/runtime/compare/1473deaa50785b956edd7d078e68c0581c1b4d95...0a940f90ebc8465791870799dd496d1a524a5381) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in SeekUnroll Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- |
  • [Test - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 1.27 secs | 1.43 secs | 1.13 | 0.00 | False | | | |
  • [Test - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 1.60 secs | 1.75 secs | 1.10 | 0.00 | False | | | |
  • [Test - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 1.28 secs | 1.43 secs | 1.12 | 0.00 | False | | | |
  • [Test - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 1.27 secs | 1.43 secs | 1.12 | 0.00 | False | | | |
  • [Test - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 1.27 secs | 1.43 secs | 1.12 | 0.00 | False | | | ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SeekUnroll*' ```
### Payloads [Baseline]() [Compare]() ### SeekUnroll.Test(boxedIndex: 19) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 11) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 27) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 3) #### ETL Files #### Histogram #### JIT Disasms ### SeekUnroll.Test(boxedIndex: 1) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | arm64 OS | Windows 10.0.19041 Queue | SurfaceWindows Baseline | [1473deaa50785b956edd7d078e68c0581c1b4d95](https://github.com/dotnet/runtime/commit/1473deaa50785b956edd7d078e68c0581c1b4d95) Compare | [0a940f90ebc8465791870799dd496d1a524a5381](https://github.com/dotnet/runtime/commit/0a940f90ebc8465791870799dd496d1a524a5381) Diff | [Diff](https://github.com/dotnet/runtime/compare/1473deaa50785b956edd7d078e68c0581c1b4d95...0a940f90ebc8465791870799dd496d1a524a5381) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.MathBenchmarks.Double Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- |
  • [AtanPi - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 62.96 μs | 67.24 μs | 1.07 | 0.02 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*' ```
### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Double.AtanPi #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | arm64 OS | Windows 10.0.19041 Queue | SurfaceWindows Baseline | [1473deaa50785b956edd7d078e68c0581c1b4d95](https://github.com/dotnet/runtime/commit/1473deaa50785b956edd7d078e68c0581c1b4d95) Compare | [0a940f90ebc8465791870799dd496d1a524a5381](https://github.com/dotnet/runtime/commit/0a940f90ebc8465791870799dd496d1a524a5381) Diff | [Diff](https://github.com/dotnet/runtime/compare/1473deaa50785b956edd7d078e68c0581c1b4d95...0a940f90ebc8465791870799dd496d1a524a5381) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.MathBenchmarks.Single Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- |
  • [AtanPi - Duration of single invocation]()
  • 📝 - [Benchmark Source]()
  • [📈 - ADX Test Multi Config Graph]()
| 52.52 μs | 57.51 μs | 1.10 | 0.04 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*' ```
### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Single.AtanPi #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Author: performanceautofiler[bot]
Assignees: -
Labels: `arch-arm64`, `os-windows`, `area-CodeGen-coreclr`, `untriaged`, `runtime-coreclr`
Milestone: -
EgorBo commented 6 months ago

Commit range https://github.com/dotnet/runtime/compare/c28bec4d3d63849c9e60dee1e7174b9a180a7e55...0fae364617f03d1003af0f0b49e57876e51875d4

@jakobbotsch could it be one of your PRs?

jakobbotsch commented 6 months ago

Yes, it can definitely be the loop alignment change that moved candidate identification into the backend – some loops may not be natural anymore after the optimization phases have run. I'll take a closer look when I'm back in January.