search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 17 Regressions on 9/11/2024 5:30:59 PM #41397

Open performanceautofiler[bot] opened 1 month ago

performanceautofiler[bot] commented 1 month ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.25 ns 3.69 ns 2.94 0.81 False
0.48 ns 2.01 ns 4.15 0.66 False
3.06 ns 5.40 ns 1.77 0.48 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'
### System.Numerics.Tests.Perf_VectorOf<UInt64>.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt64>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.68 ns 4.53 ns 1.69 0.56 False
3.28 ns 6.24 ns 1.90 0.62 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'
### System.Numerics.Tests.Perf_VectorOf<Single>.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.SubtractionOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 122eb75149baeeb902f39cafb6f88be45f19f691
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
37.80 ns 40.82 ns 1.08 0.37 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'
### System.Tests.Perf_UInt32.TryParse(value: "0") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
33.59 ns 38.53 ns 1.15 0.32 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### System.Numerics.Tests.Perf_VectorOf<Int16>.DivideBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.37 ns 4.04 ns 1.70 0.66 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### System.Collections.IterateForEach<Int32>.Span(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.07 ns 8.73 ns 4.23 0.61 False
0.84 ns 3.27 ns 3.89 0.66 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'
### System.Numerics.Tests.Perf_VectorOf<SByte>.OneBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.InequalityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.21 ns 2.74 ns 2.26 0.71 False
3.28 ns 5.55 ns 1.69 0.57 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'
### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.AddOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.18 ns 4.95 ns 1.55 0.59 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'
### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.SubtractBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.70 Ξs 12.31 Ξs 1.15 0.26 False
40.05 Ξs 42.90 Ξs 1.07 0.28 False
20.14 Ξs 24.08 Ξs 1.20 0.36 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'
### System.Numerics.Tests.Perf_VectorConvert.Convert_int_float #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorConvert.Convert_double_ulong #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorConvert.Convert_double_long #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b8d7ec0ad3e73b8f21a9905fb4ab8714b27d45fd
Compare 8e8e7f7851dcac7f58d248cbdc22e9dcb7e15a69
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.13 ns 11.36 ns 1.59 0.55 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### System.Text.Perf_Ascii.IsValid_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 month ago

range is https://github.com/dotnet/runtime/compare/8351e5f0090211d321e02fb8aaaa22d1404c6e78...a3365e4c442a728bc82e59d5583cf83bc6d31d72

not clear, what can be causing it. the only mono change is https://github.com/dotnet/runtime/pull/107464 which shouldn't change the generated code

kg commented 1 month ago

https://github.com/dotnet/runtime/commit/8351e5f0090211d321e02fb8aaaa22d1404c6e78 is in the diff range but I wouldn't expect it to impact AOT for these benchmark scenarios i.e. IsValid_Chars, they should be entirely AOT'd and not jiterpreted