search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.29k stars 4.74k forks source link

[mono][interp][Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM #109375

Open performanceautofiler[bot] opened 4 months ago

performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
22.83 ns 116.75 ns 5.11 0.20 True
15.96 ns 93.12 ns 5.83 0.21 True
25.43 ns 109.34 ns 4.30 0.20 True
18.68 ns 23.17 ns 1.24 0.28 False
19.78 ns 151.08 ns 7.64 0.25 True
16.78 ns 27.63 ns 1.65 0.26 False
18.00 ns 101.79 ns 5.66 0.28 True
47.02 ns 235.60 ns 5.01 0.20 True
12.34 ns 20.89 ns 1.69 0.23 True
30.50 ns 190.08 ns 6.23 0.11 True
42.05 ns 319.84 ns 7.61 0.12 True
17.42 ns 97.88 ns 5.62 0.22 True
18.72 ns 154.32 ns 8.24 0.23 True
8.02 ns 70.05 ns 8.74 0.40 True
14.88 ns 89.70 ns 6.03 0.34 True
21.43 ns 98.72 ns 4.61 0.26 True
15.73 ns 24.97 ns 1.59 0.30 False
15.79 ns 21.56 ns 1.37 0.32 False
55.57 ns 318.87 ns 5.74 0.16 True
16.04 ns 23.33 ns 1.46 0.32 False
15.77 ns 52.41 ns 3.32 0.20 True
14.44 ns 90.73 ns 6.28 0.29 True
27.16 ns 111.43 ns 4.10 0.14 True
16.49 ns 23.63 ns 1.43 0.27 False
18.14 ns 103.09 ns 5.68 0.25 True
15.98 ns 22.50 ns 1.41 0.34 False
11.63 ns 101.90 ns 8.76 0.34 True
45.86 ns 319.84 ns 6.97 0.08 True
16.84 ns 22.45 ns 1.33 0.34 False
9.13 ns 69.76 ns 7.64 0.39 True
36.95 ns 364.80 ns 9.87 0.15 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.LengthBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.EqualityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.ReflectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.LengthSquaredBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DistanceBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DivideByVector3OperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.InequalityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector3.ClampBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
60.65 ns 91.36 ns 1.51 0.18 False
51.03 ns 55.79 ns 1.09 0.11 False
150.85 ns 167.98 ns 1.11 0.12 False
489.49 ns 1.40 μs 2.85 0.05 True
62.05 ns 89.70 ns 1.45 0.14 False
207.97 ns 675.51 ns 3.25 0.09 True
287.49 ns 1.73 μs 6.03 0.04 True
34.06 ns 41.57 ns 1.22 0.14 False
312.27 ns 863.37 ns 2.76 0.13 True
37.47 ns 44.04 ns 1.18 0.13 False
61.66 ns 90.14 ns 1.46 0.13 False
281.95 ns 1.72 μs 6.11 0.05 True
33.79 ns 39.71 ns 1.18 0.21 False
52.08 ns 57.35 ns 1.10 0.14 False
296.42 ns 1.04 μs 3.52 0.07 True
52.18 ns 59.16 ns 1.13 0.16 False
75.11 ns 98.74 ns 1.31 0.13 False
247.56 ns 849.96 ns 3.43 0.09 True
304.77 ns 889.08 ns 2.92 0.10 True
60.97 ns 420.03 ns 6.89 0.16 True
37.10 ns 40.46 ns 1.09 0.22 False
63.25 ns 407.06 ns 6.44 0.19 True
231.11 ns 686.58 ns 2.97 0.08 True
95.15 ns 833.03 ns 8.75 0.06 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateReflectionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromVectorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
89.69 ns 365.51 ns 4.08 0.09 True
131.70 ns 145.71 ns 1.11 0.06 False
101.21 ns 513.73 ns 5.08 0.09 True
51.75 ns 74.11 ns 1.43 0.11 False
48.72 ns 73.87 ns 1.52 0.11 True
121.89 ns 154.07 ns 1.26 0.11 False
122.70 ns 280.36 ns 2.28 0.12 True
123.29 ns 152.86 ns 1.24 0.11 False
121.68 ns 145.53 ns 1.20 0.11 False
90.45 ns 350.79 ns 3.88 0.10 True
72.57 ns 490.09 ns 6.75 0.09 True
102.51 ns 524.29 ns 5.11 0.12 True
47.85 ns 77.41 ns 1.62 0.12 True
82.18 ns 96.33 ns 1.17 0.09 False
173.46 ns 750.67 ns 4.33 0.11 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.EqualityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.IsIdentityBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 87cd2c4257bd21e84d3d6990f55465be01f63399
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.95 ns 108.16 ns 9.88 0.34 True
12.88 ns 21.61 ns 1.68 0.30 True
15.44 ns 111.62 ns 7.23 0.38 True
28.94 ns 358.44 ns 12.38 0.17 True
14.27 ns 98.51 ns 6.90 0.31 True
24.51 ns 218.92 ns 8.93 0.22 True
73.27 ns 264.63 ns 3.61 0.13 True
12.75 ns 21.57 ns 1.69 0.33 False
13.00 ns 23.67 ns 1.82 0.36 False
15.37 ns 156.81 ns 10.20 0.31 True
16.16 ns 150.09 ns 9.29 0.28 True
12.30 ns 21.80 ns 1.77 0.36 False
12.83 ns 21.89 ns 1.71 0.28 False
14.40 ns 101.38 ns 7.04 0.28 True
13.60 ns 21.78 ns 1.60 0.34 False
19.62 ns 130.10 ns 6.63 0.21 True
16.74 ns 70.86 ns 4.23 0.24 True
7.91 ns 86.59 ns 10.95 0.41 True
14.85 ns 109.66 ns 7.39 0.27 True
39.30 ns 223.56 ns 5.69 0.13 True
44.72 ns 338.55 ns 7.57 0.15 True
37.98 ns 232.58 ns 6.12 0.19 True
34.21 ns 218.94 ns 6.40 0.13 True
17.56 ns 125.88 ns 7.17 0.30 True
13.42 ns 21.47 ns 1.60 0.31 False
9.18 ns 101.51 ns 11.05 0.41 True
10.35 ns 107.28 ns 10.37 0.47 True
79.75 ns 290.97 ns 3.65 0.11 True
12.82 ns 21.95 ns 1.71 0.31 False
9.00 ns 87.44 ns 9.71 0.36 True
15.37 ns 102.50 ns 6.67 0.37 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Vector2.LengthSquaredBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.SubtractOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.LengthBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.ClampBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.NormalizeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.AddFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.SubtractFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.AddOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DivideByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DivideByVector2OperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DistanceBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.EqualityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.ReflectBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DistanceSquaredBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DivideByVector2Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.MultiplyFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.InequalityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector2.DivideByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 4 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 87cd2c4257bd21e84d3d6990f55465be01f63399
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.24 ns 192.68 ns 14.55 0.20 True
7.35 ns 94.10 ns 12.81 0.36 True
6.39 ns 87.00 ns 13.61 0.34 True
6.70 ns 101.25 ns 15.11 0.33 True
50.39 ns 135.34 ns 2.69 0.17 False
0.89 ns 3.83 ns 4.32 0.92 False
35.71 ns 221.29 ns 6.20 0.15 True
6.86 ns 89.81 ns 13.09 0.33 True
43.67 ns 317.48 ns 7.27 0.11 True
42.52 ns 318.11 ns 7.48 0.14 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Vector4.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DivideByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.UnitXBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in SIMD.RayTracerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.85 secs 4.21 secs 2.27 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SIMD.RayTracerBench*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SIMD.RayTracerBench*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SIMD.RayTracerBench*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SIMD.RayTracerBench*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### SIMD.RayTracerBench.Bench #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
68.13 ns 98.22 ns 1.44 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Quaternion*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Quaternion*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Quaternion.CreateFromYawPitchRollBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
112.47 ns 408.41 ns 3.63 0.15 True
26.19 ns 98.25 ns 3.75 0.25 True
25.69 ns 97.51 ns 3.80 0.26 True
20.84 ns 94.50 ns 4.53 0.24 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Plane*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Plane*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.NormalizeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.DotNormalBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kotlarmilos commented 4 months ago

Regressions introduced in https://github.com/dotnet/runtime/pull/103462.

dotnet-policy-service[bot] commented 1 week ago

Tagging subscribers to this area: @fanyang-mono, @steveisok See info in area-owners.md if you want to be subscribed.

matouskozak commented 1 week ago

Possibly introduced in https://github.com/dotnet/runtime/pull/103462, cc @tannergooding

tannergooding commented 1 week ago

Looks like these are all centered around Vector3 which Mono doesn't have accelerated on all platforms.

This probably isn't terribly difficult to handle for most of the common scenarios.