dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 13 Regressions on 8/8/2024 11:52:00 PM #39850

Open performanceautofiler[bot] opened 2 months ago

performanceautofiler[bot] commented 2 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 44b6b2ae056092d7759b068a9661b86627ec1024
Compare 48aa31535f0e5f318f061eb6f8ad4f25611e6133
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
22.27 ns 24.77 ns 1.11 0.35 False
6.31 ns 9.74 ns 1.54 0.44 False
6.87 ns 10.67 ns 1.55 0.51 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### System.Memory.Span<Int32>.SequenceCompareTo(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Span<Int32>.LastIndexOfAnyValues(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfValue(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 44b6b2ae056092d7759b068a9661b86627ec1024
Compare 48aa31535f0e5f318f061eb6f8ad4f25611e6133
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
34.68 ns 45.08 ns 1.30 0.26 False
19.02 ns 24.12 ns 1.27 0.38 False
22.20 ns 26.93 ns 1.21 0.35 False
19.94 ns 27.50 ns 1.38 0.40 False
26.65 ns 31.52 ns 1.18 0.31 False
22.38 ns 28.05 ns 1.25 0.36 False
29.34 ns 32.39 ns 1.10 0.46 False
29.62 ns 38.20 ns 1.29 0.49 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 259 bytes, DiffLastByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, DiffFirstByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, DiffLastByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, DiffMiddleByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 259 bytes, DiffMiddleByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, Same) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 16,16 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 16,8 bits) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 44b6b2ae056092d7759b068a9661b86627ec1024
Compare 48aa31535f0e5f318f061eb6f8ad4f25611e6133
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
32.81 ns 41.80 ns 1.27 0.47 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
### System.Tests.Perf_Random.Next_long_unseeded #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 44b6b2ae056092d7759b068a9661b86627ec1024
Compare 48aa31535f0e5f318f061eb6f8ad4f25611e6133
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.87 ns 10.46 ns 1.33 0.43 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 2 months ago

Range is https://github.com/dotnet/runtime/compare/105bf4dd4dedfe9dfb067427cbca34f76ddfaa44...68511fd27fe4055ce5203742998ba12019dfcbd4

Looks like https://github.com/dotnet/runtime/pull/106080, @kg is that expected?

kg commented 2 months ago

Looks like dotnet/runtime#106080, @kg is that expected?

No, and it seems kind of arbitrary for it to only impact these.