search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 207 Improvements on 2/21/2024 11:59:23 AM #29923

Closed performanceautofiler[bot] closed 4 months ago

performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.65 ns 0.75 ns 0.28 0.05 True
9.59 ns 3.28 ns 0.34 0.01 True
6.46 ns 0.29 ns 0.05 0.06 True
13.51 ns 3.18 ns 0.24 0.02 True
9.26 ns 3.09 ns 0.33 0.00 True
13.71 ns 3.18 ns 0.23 0.01 True
6.64 ns 0.35 ns 0.05 0.06 True
6.61 ns 0.35 ns 0.05 0.11 True
12.79 ns 3.18 ns 0.25 0.01 True
6.28 ns 0.35 ns 0.05 0.09 True
2.87 ns 0.31 ns 0.11 0.14 True
2.41 ns 0.44 ns 0.18 0.09 True
7.58 ns 0.34 ns 0.04 0.10 True
13.73 ns 3.03 ns 0.22 0.00 True
9.27 ns 2.72 ns 0.29 0.03 True
33.03 ns 28.15 ns 0.85 0.04 False
2.43 ns 0.35 ns 0.14 0.07 True
13.25 ns 3.15 ns 0.24 0.00 True
9.26 ns 2.71 ns 0.29 0.02 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<UInt32>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<UInt32>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
24.96 ns 0.36 ns 0.01 0.04 True
24.28 ns 18.30 ns 0.75 0.01 True
40.58 ns 3.18 ns 0.08 0.02 True
52.23 ns 42.87 ns 0.82 0.02 True
38.61 ns 3.12 ns 0.08 0.02 True
2.55 ns 0.35 ns 0.14 0.10 True
2.59 ns 0.41 ns 0.16 0.03 True
24.61 ns 0.36 ns 0.01 0.03 True
9.09 ns 3.15 ns 0.35 0.01 True
9.31 ns 2.85 ns 0.31 0.01 True
50.78 ns 42.85 ns 0.84 0.02 True
25.01 ns 0.30 ns 0.01 0.04 True
2.52 ns 0.32 ns 0.13 0.07 True
27.98 ns 24.32 ns 0.87 0.01 False
36.18 ns 3.02 ns 0.08 0.01 True
30.85 ns 0.34 ns 0.01 0.06 True
41.59 ns 3.20 ns 0.08 0.01 True
44.33 ns 39.93 ns 0.90 0.02 False
27.53 ns 24.22 ns 0.88 0.01 False
9.61 ns 3.29 ns 0.34 0.01 True
42.79 ns 3.04 ns 0.07 0.01 True
31.21 ns 0.40 ns 0.01 0.07 True
2.65 ns 0.44 ns 0.17 0.06 True
34.33 ns 29.51 ns 0.86 0.01 False
9.04 ns 2.76 ns 0.31 0.03 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Byte>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Byte>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.DivideBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.DivisionOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.82 ns 3.14 ns 0.25 0.01 True
10.44 ns 2.89 ns 0.28 0.03 True
3.28 ns 0.35 ns 0.11 0.06 True
13.78 ns 3.18 ns 0.23 0.01 True
13.50 ns 3.18 ns 0.24 0.00 True
9.28 ns 2.71 ns 0.29 0.03 True
9.64 ns 3.29 ns 0.34 0.00 True
13.36 ns 3.17 ns 0.24 0.01 True
13.52 ns 3.20 ns 0.24 0.01 True
9.19 ns 3.18 ns 0.35 0.00 True
13.74 ns 3.11 ns 0.23 0.00 True
5.96 ns 0.36 ns 0.06 0.17 True
32.82 ns 28.19 ns 0.86 0.03 False
2.40 ns 0.33 ns 0.14 0.03 True
2.24 ns 0.35 ns 0.16 0.06 True
2.40 ns 0.31 ns 0.13 0.04 True
9.26 ns 2.72 ns 0.29 0.01 True
6.50 ns 0.32 ns 0.05 0.04 True
13.73 ns 3.17 ns 0.23 0.01 True
6.46 ns 0.34 ns 0.05 0.05 True
6.40 ns 0.31 ns 0.05 0.07 True
6.34 ns 0.35 ns 0.06 0.09 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Int32>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Int32>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.32 ns 0.30 ns 0.02 0.07 True
20.83 ns 0.36 ns 0.02 0.05 True
2.60 ns 0.34 ns 0.13 0.05 True
17.66 ns 0.30 ns 0.02 0.06 True
27.68 ns 3.19 ns 0.12 0.02 True
9.26 ns 3.18 ns 0.34 0.02 True
23.24 ns 3.18 ns 0.14 0.01 True
9.26 ns 2.70 ns 0.29 0.02 True
14.88 ns 0.31 ns 0.02 0.06 True
28.65 ns 3.19 ns 0.11 0.01 True
23.26 ns 3.15 ns 0.14 0.01 True
1.74 ns 0.61 ns 0.35 0.12 True
3.39 ns 0.36 ns 0.10 0.13 True
3.40 ns 0.32 ns 0.09 0.09 True
27.92 ns 3.18 ns 0.11 0.01 True
50.72 ns 45.96 ns 0.91 0.03 False
9.17 ns 2.80 ns 0.31 0.02 True
59.14 ns 52.50 ns 0.89 0.09 False
20.88 ns 0.31 ns 0.01 0.03 True
2.25 ns 0.31 ns 0.14 0.07 True
9.60 ns 3.27 ns 0.34 0.00 True
18.65 ns 2.90 ns 0.16 0.03 True
22.70 ns 3.18 ns 0.14 0.01 True
27.48 ns 3.18 ns 0.12 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int16>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Int16>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int16>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Int16>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.25 ns 2.85 ns 0.31 0.02 True
24.79 ns 0.32 ns 0.01 0.02 True
27.53 ns 24.24 ns 0.88 0.01 True
2.49 ns 0.32 ns 0.13 0.07 True
34.94 ns 3.19 ns 0.09 0.01 True
2.50 ns 0.31 ns 0.12 0.08 True
55.77 ns 42.35 ns 0.76 0.02 True
9.58 ns 3.28 ns 0.34 0.00 True
25.19 ns 0.34 ns 0.01 0.04 True
34.52 ns 3.18 ns 0.09 0.01 True
2.30 ns 0.31 ns 0.14 0.08 True
27.54 ns 2.64 ns 0.10 0.03 True
31.12 ns 0.36 ns 0.01 0.04 True
1.61 ns 0.60 ns 0.37 0.10 True
33.32 ns 3.19 ns 0.10 0.01 True
27.95 ns 0.35 ns 0.01 0.04 True
34.35 ns 3.19 ns 0.09 0.01 True
35.09 ns 3.18 ns 0.09 0.02 True
43.80 ns 3.15 ns 0.07 0.01 True
24.43 ns 0.34 ns 0.01 0.02 True
44.61 ns 3.18 ns 0.07 0.01 True
2.61 ns 0.36 ns 0.14 0.06 True
29.45 ns 24.81 ns 0.84 0.01 True
24.21 ns 19.05 ns 0.79 0.01 True
9.32 ns 2.88 ns 0.31 0.02 True
8.86 ns 3.18 ns 0.36 0.00 True
55.87 ns 42.36 ns 0.76 0.03 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<SByte>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<SByte>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.DivideBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<SByte>.DivisionOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.26 ns 3.08 ns 0.30 0.01 True
1.70 ns 0.61 ns 0.36 0.11 False
10.04 ns 3.19 ns 0.32 0.01 True
3.31 ns 0.35 ns 0.11 0.08 True
9.29 ns 2.85 ns 0.31 0.01 True
4.49 ns 0.31 ns 0.07 0.04 True
2.55 ns 0.31 ns 0.12 0.09 True
9.17 ns 3.17 ns 0.35 0.01 True
21.98 ns 17.03 ns 0.77 0.05 True
10.29 ns 3.18 ns 0.31 0.00 True
9.84 ns 3.18 ns 0.32 0.01 True
9.29 ns 2.85 ns 0.31 0.03 True
2.42 ns 0.36 ns 0.15 0.04 True
9.58 ns 3.29 ns 0.34 0.01 True
3.35 ns 0.32 ns 0.09 0.06 True
2.60 ns 0.30 ns 0.12 0.07 True
3.74 ns 0.34 ns 0.09 0.07 True
4.52 ns 0.30 ns 0.07 0.05 True
10.14 ns 3.17 ns 0.31 0.01 True
8.50 ns 3.10 ns 0.36 0.01 True
10.03 ns 3.19 ns 0.32 0.00 True
19.07 ns 17.41 ns 0.91 0.03 False
10.06 ns 3.18 ns 0.32 0.01 True
2.50 ns 0.35 ns 0.14 0.05 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int64>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Int64>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int64>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Int64>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int64>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.DivideBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
21.23 ns 3.16 ns 0.15 0.01 True
9.99 ns 0.31 ns 0.03 0.08 True
9.63 ns 3.20 ns 0.33 0.01 True
9.66 ns 3.11 ns 0.32 0.00 True
9.69 ns 0.32 ns 0.03 0.09 True
15.38 ns 3.18 ns 0.21 0.00 True
9.31 ns 2.86 ns 0.31 0.02 True
9.29 ns 2.86 ns 0.31 0.01 True
15.21 ns 0.46 ns 0.03 0.06 True
3.43 ns 0.43 ns 0.12 0.08 True
17.48 ns 3.20 ns 0.18 0.01 True
17.55 ns 3.19 ns 0.18 0.01 True
35.22 ns 28.27 ns 0.80 0.03 False
17.25 ns 3.18 ns 0.18 0.02 True
28.62 ns 3.29 ns 0.12 0.00 True
66.70 ns 1.60 ns 0.02 0.04 True
9.67 ns 0.31 ns 0.03 0.08 True
3.66 ns 0.96 ns 0.26 0.11 True
9.60 ns 3.20 ns 0.33 0.01 True
16.59 ns 3.18 ns 0.19 0.00 True
16.58 ns 3.16 ns 0.19 0.01 True
9.60 ns 3.20 ns 0.33 0.01 True
3.14 ns 0.31 ns 0.10 0.06 True
10.41 ns 0.30 ns 0.03 0.09 True
9.59 ns 3.19 ns 0.33 0.01 True
20.61 ns 3.20 ns 0.16 0.01 True
2.73 ns 0.29 ns 0.11 0.05 True
9.13 ns 0.31 ns 0.03 0.10 True
26.21 ns 3.20 ns 0.12 0.01 True
9.64 ns 3.11 ns 0.32 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Single>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Single>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseOrBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.XorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseOrOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseAndBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Single>.ExclusiveOrOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.31 ns 0.36 ns 0.08 0.06 True
9.29 ns 2.71 ns 0.29 0.02 True
2.47 ns 0.31 ns 0.13 0.04 True
3.42 ns 0.31 ns 0.09 0.11 True
9.63 ns 3.19 ns 0.33 0.00 True
2.49 ns 0.31 ns 0.13 0.04 True
9.58 ns 3.19 ns 0.33 0.01 True
10.66 ns 3.20 ns 0.30 0.02 True
4.44 ns 0.34 ns 0.08 0.08 True
9.65 ns 3.20 ns 0.33 0.01 True
3.77 ns 0.57 ns 0.15 0.03 True
2.75 ns 0.34 ns 0.12 0.08 True
11.34 ns 3.19 ns 0.28 0.01 True
3.49 ns 0.30 ns 0.09 0.10 True
10.73 ns 3.18 ns 0.30 0.01 True
3.48 ns 0.34 ns 0.10 0.08 True
13.20 ns 3.18 ns 0.24 0.02 True
3.01 ns 0.33 ns 0.11 0.08 True
10.70 ns 3.11 ns 0.29 0.01 True
9.61 ns 3.19 ns 0.33 0.00 True
10.47 ns 9.40 ns 0.90 0.00 True
8.76 ns 2.96 ns 0.34 0.01 True
10.75 ns 3.20 ns 0.30 0.00 True
10.77 ns 3.20 ns 0.30 0.00 True
10.93 ns 3.14 ns 0.29 0.01 True
23.64 ns 19.20 ns 0.81 0.07 False
9.35 ns 2.71 ns 0.29 0.01 True
15.43 ns 3.19 ns 0.21 0.00 True
43.69 ns 1.01 ns 0.02 0.02 True
28.79 ns 3.14 ns 0.11 0.01 True
9.58 ns 3.19 ns 0.33 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Double>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Double>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.EqualsAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.ExclusiveOrOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.XorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseOrOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanAllBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualAnyBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseOrBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.EqualsStaticBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.AndNotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseAndBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 11bdd1b9fc38e137d7945c66790d05b0cd1966ab
Compare cefd1a7ad9ec163715972685924519fe5cb7bf61
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.13 μs 11.39 μs 0.67 0.01 True
8.53 μs 2.42 μs 0.28 0.00 True
17.79 μs 11.83 μs 0.67 0.01 True
30.14 μs 18.44 μs 0.61 0.00 True
17.45 μs 11.66 μs 0.67 0.00 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorConvert.Convert_long_double #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorConvert.Convert_int_float #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorConvert.Convert_ulong_double #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorConvert.Widen_float #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_VectorConvert.Narrow_double #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
matouskozak commented 4 months ago

https://github.com/dotnet/runtime/pull/98317 @vargaz , good job!