search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 2035 Regressions on 2/20/2024 2:11:38 AM #30141

Closed performanceautofiler[bot] closed 7 months ago

performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.93 ns 47.08 ns 2.95 0.10 True
34.84 ns 42.48 ns 1.22 0.03 False
15.06 ns 18.39 ns 1.22 0.22 False
123.07 ns 140.60 ns 1.14 0.00 True
42.93 ns 82.28 ns 1.92 0.08 True
110.15 ns 272.46 ns 2.47 0.11 True
30.18 ns 152.17 ns 5.04 0.06 True
16.32 ns 24.53 ns 1.50 0.02 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_UInt64*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_UInt64* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_UInt64*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_UInt64* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_UInt64.ToString(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.TryFormat(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.TryParseHex(value: "0") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.TryParseHex(value: "3039") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_UInt64.TryFormat(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.96 ns 11.30 ns 1.26 0.15 False
33.31 ns 104.24 ns 3.13 0.05 True
5.29 ns 15.90 ns 3.01 0.11 True
5.75 ns 16.78 ns 2.92 0.14 True
9.08 ns 18.23 ns 2.01 0.11 True
31.90 ns 103.49 ns 3.24 0.08 True
5.79 ns 16.75 ns 2.89 0.15 True
25.70 ns 64.20 ns 2.50 0.09 True
5.29 ns 15.83 ns 2.99 0.17 True
9.12 ns 18.20 ns 2.00 0.13 True
7.62 ns 17.29 ns 2.27 0.11 True
7.51 ns 17.24 ns 2.29 0.12 True

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Boolean*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Boolean* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Boolean*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Boolean* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Boolean.TryParse(value: "0") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: " True ") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: "true") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: "true") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: "False") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: " True ") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: "TRUE") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: "Bogus") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: "TRUE") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: "false") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: "False") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: "false") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.Perf_DefaultFrozenDictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
139.02 μs 158.68 μs 1.14 0.01 False
149.60 ns 172.74 ns 1.15 0.02 False
1.41 μs 1.69 μs 1.20 0.06 True
316.38 μs 444.44 μs 1.40 0.01 True
28.98 μs 39.20 μs 1.35 0.02 True
390.82 μs 564.12 μs 1.44 0.02 False
2.49 μs 3.76 μs 1.51 0.02 True
20.72 μs 24.62 μs 1.19 0.00 True
1.43 ms 1.58 ms 1.11 0.02 False
1.78 μs 2.38 μs 1.34 0.03 False
33.80 μs 44.32 μs 1.31 0.02 True

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Perf_DefaultFrozenDictionary*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Perf_DefaultFrozenDictionary* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Perf_DefaultFrozenDictionary*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Perf_DefaultFrozenDictionary* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.Perf_DefaultFrozenDictionary.ToFrozenDictionary(Count: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_True_FrozenDictionary(Count: 10) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.ToFrozenDictionary(Count: 10) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 10000) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_True_FrozenDictionary(Count: 10000) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_True_FrozenDictionary(Count: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.ToFrozenDictionary(Count: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.ToFrozenDictionary(Count: 10000) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Perf_DefaultFrozenDictionary.TryGetValue_True_FrozenDictionary(Count: 1000) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.76 μs 12.26 μs 1.40 0.15 True
1.48 μs 1.73 μs 1.17 0.07 False
4.68 ms 7.27 ms 1.55 0.00 True
1.20 μs 1.72 μs 1.43 0.05 True
9.90 μs 13.65 μs 1.38 0.09 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Directory*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.Perf_Directory* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Directory*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.Perf_Directory* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.IO.Tests.Perf_Directory.MoveFiles #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_Directory.Exists #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_Directory.EnumerateFiles #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_Directory.GetCurrentDirectory #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_Directory.MoveFolders #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
153.16 ns 162.94 ns 1.06 0.00 False
223.84 ns 334.16 ns 1.49 0.02 True
955.75 μs 2.01 ms 2.10 0.00 True
140.61 ns 272.38 ns 1.94 0.02 True
288.26 ns 412.31 ns 1.43 0.02 True
137.89 ns 266.76 ns 1.93 0.02 True
288.45 ns 462.84 ns 1.60 0.02 True
154.92 ns 165.54 ns 1.07 0.00 False
241.96 ns 269.60 ns 1.11 0.03 True
26.90 μs 58.39 μs 2.17 0.08 True
239.31 ns 542.26 ns 2.27 0.03 True
26.89 μs 58.33 μs 2.17 0.08 True
248.54 ns 264.64 ns 1.06 0.00 True
285.42 ns 456.66 ns 1.60 0.02 True
296.82 ns 469.60 ns 1.58 0.03 True
241.81 ns 267.26 ns 1.11 0.03 True
143.24 ns 270.82 ns 1.89 0.01 True
141.71 ns 268.88 ns 1.90 0.03 True
297.30 ns 469.79 ns 1.58 0.03 True
210.65 ns 318.44 ns 1.51 0.01 True
209.56 ns 320.37 ns 1.53 0.03 True
274.47 ns 405.90 ns 1.48 0.02 True
248.21 ns 265.93 ns 1.07 0.01 True
300.67 ns 465.88 ns 1.55 0.04 True
486.37 ns 753.20 ns 1.55 0.01 True
295.33 ns 466.15 ns 1.58 0.04 True
287.80 ns 457.83 ns 1.59 0.03 True
289.96 ns 458.03 ns 1.58 0.03 True
951.59 μs 1.97 ms 2.07 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Double*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Double* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Double*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Double* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Double.TryParse(value: "12345") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.IsNaN(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.Parse(value: "12345") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "F50") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "F50") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToString(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "F50") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Double.IsNaN(value: NaN) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
108.95 μs 153.17 μs 1.41 0.01 False
3.42 ns 4.60 ns 1.34 0.12 False
153.25 μs 240.39 μs 1.57 0.07 False
43.83 μs 109.54 μs 2.50 0.02 False
65.53 μs 109.50 μs 1.67 0.02 False
65.71 μs 109.45 μs 1.67 0.01 False
181.28 μs 218.69 μs 1.21 0.09 False
131.00 μs 218.60 μs 1.67 0.05 True
170.36 μs 350.52 μs 2.06 0.06 True
181.44 μs 240.98 μs 1.33 0.08 False
65.75 μs 109.57 μs 1.67 0.01 False
87.41 μs 219.22 μs 2.51 0.02 True
152.84 μs 240.30 μs 1.57 0.05 True
43.62 μs 109.80 μs 2.52 0.01 False
109.49 μs 218.96 μs 2.00 0.03 False
43.76 μs 109.50 μs 2.50 0.02 False
43.65 μs 152.53 μs 3.49 0.03 False
147.45 μs 241.27 μs 1.64 0.07 False
65.58 μs 109.31 μs 1.67 0.02 False
180.83 μs 241.17 μs 1.33 0.08 False
43.76 μs 153.24 μs 3.50 0.04 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'PerfLabTests.CastingPerf*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter PerfLabTests.CastingPerf* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'PerfLabTests.CastingPerf*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter PerfLabTests.CastingPerf* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterfaceNo #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjrefValueTypeObj #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjFooIsObj #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjInt #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.IFooFooIsIFoo #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.FooObjIsFoo #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjObjIsFoo #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.FooObjCastIfIsa #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.IntObj #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjFooIsObj2 #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjScalarValueType #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.CheckObjIsInterfaceYes #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.FooObjIsNull #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ScalarValueTypeObj #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.ObjObjrefValueType #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.FooObjIsFoo2 #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.CastingPerf.CheckObjIsInterfaceNo #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
627.12 ns 1.54 μs 2.45 0.06 True
561.30 ns 1.54 μs 2.74 0.07 False
412.99 ns 885.87 ns 2.15 0.06 True
450.05 ns 1.76 μs 3.91 0.05 True
447.82 ns 888.43 ns 1.98 0.02 True
450.36 ns 1.11 μs 2.46 0.06 True
418.57 ns 883.91 ns 2.11 0.07 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_BitOperations* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_BitOperations* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Numerics.Tests.Perf_BitOperations.Log2_ulong #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BitOperations.Log2_uint #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BitOperations.TrailingZeroCount_ulong #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BitOperations.PopCount_ulong #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BitOperations.TrailingZeroCount_uint #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
527.97 μs 674.00 μs 1.28 0.03 False
538.56 μs 680.99 μs 1.26 0.03 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Benchmark.GetChildKeysTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Benchmark.GetChildKeysTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Benchmark.GetChildKeysTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Benchmark.GetChildKeysTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### Benchmark.GetChildKeysTests.AddChainedConfigurationWithCommonPaths #### ETL Files #### Histogram #### JIT Disasms ### Benchmark.GetChildKeysTests.AddChainedConfigurationWithSplitting #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.02 ns 456.82 ns 225.89 0.13 True
0.49 ns 456.39 ns 922.45 0.26 True
0.42 ns 681.76 ns 1616.78 0.46 True
2.70 μs 4.38 μs 1.62 0.06 False
1.36 μs 3.19 μs 2.34 0.14 True
18.38 μs 26.27 μs 1.43 0.04 True
17.25 μs 26.36 μs 1.53 0.04 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IterateFor<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.IterateFor<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IterateFor<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.IterateFor<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.IterateFor<String>.Span(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateFor<String>.Array(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateFor<String>.List(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateFor<String>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in Benchstone.BenchI.BubbleSort2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.71 ms 77.57 ms 4.15 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Benchstone.BenchI.BubbleSort2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Benchstone.BenchI.BubbleSort2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Benchstone.BenchI.BubbleSort2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Benchstone.BenchI.BubbleSort2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### Benchstone.BenchI.BubbleSort2.Test #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.61 μs 11.49 μs 1.74 0.00 True
54.09 μs 66.95 μs 1.24 0.03 True
73.94 μs 97.71 μs 1.32 0.08 True
22.92 μs 43.97 μs 1.92 0.10 True
5.89 μs 15.30 μs 2.60 0.02 True
5.40 μs 11.62 μs 2.15 0.00 True
23.92 μs 43.71 μs 1.83 0.05 True
4.49 μs 9.82 μs 2.19 0.00 True
86.59 μs 142.67 μs 1.65 0.03 True
5.06 μs 32.38 μs 6.40 0.05 True
5.39 μs 11.62 μs 2.16 0.00 True
39.27 μs 61.05 μs 1.55 0.05 True
37.33 μs 51.32 μs 1.37 0.03 True
4.44 μs 11.56 μs 2.60 0.00 True
36.49 μs 57.68 μs 1.58 0.07 True
54.70 μs 68.49 μs 1.25 0.04 True
27.43 μs 95.66 μs 3.49 0.02 True
30.93 μs 50.18 μs 1.62 0.12 True
14.27 μs 17.85 μs 1.25 0.09 False
13.34 μs 18.70 μs 1.40 0.12 False
25.00 μs 51.24 μs 2.05 0.07 True
74.11 μs 95.44 μs 1.29 0.04 True
36.91 μs 56.78 μs 1.54 0.11 True
23.00 μs 47.71 μs 2.07 0.10 True
54.08 μs 75.43 μs 1.39 0.01 True
58.82 μs 123.48 μs 2.10 0.15 True
4.81 μs 16.46 μs 3.42 0.02 True
41.35 μs 66.44 μs 1.61 0.07 True
73.79 μs 93.37 μs 1.27 0.00 True
81.50 μs 124.77 μs 1.53 0.00 True
55.56 μs 65.93 μs 1.19 0.12 False
7.84 μs 15.32 μs 1.95 0.00 True
38.80 μs 117.34 μs 3.02 0.05 True
27.47 μs 118.77 μs 4.32 0.02 True
55.94 μs 75.68 μs 1.35 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.MathBenchmarks.Double*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.MathBenchmarks.Double* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.MathBenchmarks.Double*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.MathBenchmarks.Double* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.MathBenchmarks.Double.Sqrt #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.AsinPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Exp10M1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Exp #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Min #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Floor #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Log #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Abs #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.SinCosPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.FusedMultiplyAdd #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Ceiling #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Log10 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Log2 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Round #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Log2P1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.AcosPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Exp2 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Sin #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.ILogB #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.ScaleB #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.LogP1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Exp10 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Cos #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.ExpM1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.SinPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.SinCos #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Max #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Log10P1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Pow #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.TanPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.AtanPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.CopySign #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Hypot #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.Exp2M1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Double.CosPi #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.71 ns 15.69 ns 2.34 0.05 True
6.07 ns 12.76 ns 2.10 0.06 True
3.95 ns 9.16 ns 2.32 0.02 True
8.38 ns 17.14 ns 2.04 0.07 True
4.61 ns 9.94 ns 2.16 0.04 True
4.13 ns 10.15 ns 2.46 0.15 True
4.95 ns 18.54 ns 3.75 0.15 True
5.09 ns 10.37 ns 2.04 0.02 True
9.03 ns 23.19 ns 2.57 0.03 True
7.03 ns 13.85 ns 1.97 0.11 True
11.86 ns 23.93 ns 2.02 0.13 True
77.64 ns 149.82 ns 1.93 0.04 True
10.46 ns 30.09 ns 2.88 0.01 True
14.05 ns 24.94 ns 1.78 0.04 True
8.93 ns 16.23 ns 1.82 0.15 True
32.73 ns 49.70 ns 1.52 0.12 False
46.34 ns 64.78 ns 1.40 0.02 True
71.20 ns 124.80 ns 1.75 0.03 True
4.65 ns 9.90 ns 2.13 0.06 True
7.65 ns 16.58 ns 2.17 0.08 True
58.59 ns 88.87 ns 1.52 0.08 False
12.62 ns 18.13 ns 1.44 0.15 False
79.31 ns 151.94 ns 1.92 0.01 True
8.19 ns 15.77 ns 1.93 0.14 True
7.82 ns 15.57 ns 1.99 0.17 True
7.82 ns 14.47 ns 1.85 0.10 True
70.58 ns 125.78 ns 1.78 0.07 True
25.84 ns 51.42 ns 1.99 0.13 False
26.60 ns 54.04 ns 2.03 0.13 False
4.65 ns 9.88 ns 2.12 0.02 True
7.89 ns 15.54 ns 1.97 0.14 True
53.56 ns 75.11 ns 1.40 0.07 True
4.66 ns 9.93 ns 2.13 0.04 True
4.19 ns 9.68 ns 2.31 0.01 True
4.64 ns 10.00 ns 2.16 0.01 True
5.60 ns 10.38 ns 1.85 0.14 True
14.22 ns 24.68 ns 1.74 0.03 True
7.12 ns 15.54 ns 2.18 0.18 True
3.95 ns 9.17 ns 2.32 0.02 True
23.49 ns 41.65 ns 1.77 0.03 True
5.68 ns 18.56 ns 3.27 0.09 True
8.44 ns 22.66 ns 2.68 0.10 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performanc

performanceautofiler[bot] commented 7 months ago

e/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Buffers.Text.Tests.Utf8ParserTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Buffers.Text.Tests.Utf8ParserTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: TRUE) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: -32768) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: 127) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 4294967295) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 2147483647) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: false) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 65535) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: -128) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: False) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: True ) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: -2147483648) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: true) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
967.28 ns 1.60 μs 1.65 0.18 True
940.66 ns 1.56 μs 1.66 0.12 True
8.28 μs 14.41 μs 1.74 0.02 True
58.53 μs 110.79 μs 1.89 0.01 True
80.15 μs 151.17 μs 1.89 0.03 True
8.00 μs 13.65 μs 1.71 0.01 True
12.02 μs 20.89 μs 1.74 0.02 True
72.66 μs 139.84 μs 1.92 0.01 True
12.77 μs 22.17 μs 1.74 0.01 True
48.67 μs 104.32 μs 2.14 0.01 True
6.99 ms 12.08 ms 1.73 0.01 True
19.27 μs 31.72 μs 1.65 0.03 True
41.16 μs 94.15 μs 2.29 0.01 True
6.56 ms 11.46 ms 1.75 0.00 True
17.44 μs 29.73 μs 1.70 0.03 True
62.44 μs 117.93 μs 1.89 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Node.Tests.Perf_ParseThenWrite* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Node.Tests.Perf_ParseThenWrite* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
129.71 μs 3.73 ms 28.78 0.17 True
476.68 ns 15.02 μs 31.51 0.08 True
228.09 μs 5.92 ms 25.94 0.11 True
46.31 ns 155.71 ns 3.36 0.08 True
477.60 ns 15.03 μs 31.47 0.08 True
16.83 ns 146.96 ns 8.73 0.13 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.BinaryWriterExtendedTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.BinaryWriterExtendedTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 32) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 32) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.39 μs 72.21 μs 16.44 0.03 True
51.05 μs 85.04 μs 1.67 0.06 False
10.96 μs 77.76 μs 7.10 0.02 True
0.00 ns 65.02 μs 41400414.45 0.41 True
4.39 μs 71.21 μs 16.24 0.02 True
6.58 μs 77.08 μs 11.71 0.03 True
50.13 μs 75.04 μs 1.50 0.05 False
15.15 μs 78.99 μs 5.21 0.04 True
10.97 μs 81.10 μs 7.40 0.02 True
15.38 μs 72.74 μs 4.73 0.08 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'IfStatements.IfStatements*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter IfStatements.IfStatements* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'IfStatements.IfStatements*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter IfStatements.IfStatements* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### IfStatements.IfStatements.And #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.OrArray #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.OrOr #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.Single #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.Or #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.AndAnd #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.AndArray #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.AndAndAnd #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.AndOr #### ETL Files #### Histogram #### JIT Disasms ### IfStatements.IfStatements.SingleArray #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in SIMD.ConsoleMandel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
745.88 ms 1.38 secs 1.85 0.00 True
750.69 ms 1.98 secs 2.63 0.00 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SIMD.ConsoleMandel*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter SIMD.ConsoleMandel* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SIMD.ConsoleMandel*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter SIMD.ConsoleMandel* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### SIMD.ConsoleMandel.ScalarDoubleSinglethreadRaw #### ETL Files #### Histogram #### JIT Disasms ### SIMD.ConsoleMandel.ScalarFloatSinglethreadRaw #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
461.43 ns 902.22 ns 1.96 0.01 True
464.05 ns 905.64 ns 1.95 0.03 True
462.01 ns 902.73 ns 1.95 0.00 True
461.97 ns 900.84 ns 1.95 0.00 True
686.38 ns 905.76 ns 1.32 0.14 False
4.53 μs 9.04 μs 2.00 0.00 True
694.08 ns 2.03 μs 2.92 0.03 False
468.55 ns 914.26 ns 1.95 0.02 False
459.01 ns 2.02 μs 4.40 0.03 True
463.88 ns 906.51 ns 1.95 0.02 True
464.14 ns 681.75 ns 1.47 0.03 True
461.30 ns 910.94 ns 1.97 0.00 True
365.50 ns 1.80 μs 4.93 0.02 True
406.23 ns 904.88 ns 2.23 0.02 True
1.81 μs 2.26 μs 1.24 0.02 True
585.43 ns 1.52 μs 2.60 0.01 True
461.92 ns 906.59 ns 1.96 0.00 True
461.54 ns 906.43 ns 1.96 0.02 False
459.56 ns 1.35 μs 2.93 0.02 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Span.IndexerBench*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Span.IndexerBench* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Span.IndexerBench*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Span.IndexerBench* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### Span.IndexerBench.ReadOnlyIndexer1(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Fixed2(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.ReadOnlyIndexer2(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Indexer3(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Indexer6(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Indexer4(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.CoveredIndex2(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.KnownSizeCtor2(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.WriteViaIndexer2(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Indexer5(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Fixed1(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Indexer2(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.WriteViaIndexer1(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.Ref(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.CoveredIndex3(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.CoveredIndex1(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.KnownSizeArray(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.KnownSizeCtor(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Span.IndexerBench.SameIndex1(length: 1024) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
26.37 ns 36.20 ns 1.37 0.07 False
17.79 ns 43.52 ns 2.45 0.12 True
689.70 ns 2.49 μs 3.61 0.10 True
12.51 μs 14.07 μs 1.12 0.01 True
325.69 ns 345.45 ns 1.06 0.09 False
66.20 ns 119.47 ns 1.80 0.11 True
49.30 ns 93.35 ns 1.89 0.01 True
628.86 ns 2.13 μs 3.39 0.08 True
273.04 ns 1.69 μs 6.17 0.00 True
330.04 ns 349.12 ns 1.06 0.09 False
629.90 ns 2.14 μs 3.39 0.08 True
16.36 ns 44.45 ns 2.72 0.09 True
15.95 ns 42.79 ns 2.68 0.14 True
16.32 ns 43.30 ns 2.65 0.17 True
625.73 ns 2.13 μs 3.40 0.08 True
15.45 ns 43.46 ns 2.81 0.11 True
322.11 ns 1.68 μs 5.22 0.08 True
15.42 ns 26.79 ns 1.74 0.18 True
629.25 ns 2.13 μs 3.38 0.08 True
272.98 ns 1.68 μs 6.15 0.00 True
54.28 ns 93.36 ns 1.72 0.10 False
627.28 ns 2.13 μs 3.40 0.08 True
682.03 ns 2.49 μs 3.65 0.10 True
272.26 ns 1.67 μs 6.15 0.00 True
68.25 ns 72.11 ns 1.06 0.10 False
295.91 ns 1.68 μs 5.67 0.00 True
15.41 ns 43.45 ns 2.82 0.08 True
628.39 ns 2.13 μs 3.39 0.08 True
35.84 ns 54.28 ns 1.51 0.08 False
272.02 ns 1.68 μs 6.18 0.00 True
12.54 μs 14.05 μs 1.12 0.00 True
297.28 ns 1.68 μs 5.66 0.00 True
683.87 ns 2.49 μs 3.65 0.12 True
27.48 ns 38.85 ns 1.41 0.08 False
698.50 ns 2.49 μs 3.57 0.10 True
12.70 μs 14.30 μs 1.13 0.01 False
15.45 ns 43.31 ns 2.80 0.12 True
16.27 ns 44.20 ns 2.72 0.13 True
324.12 ns 1.69 μs 5.20 0.08 True
17.99 ns 43.48 ns 2.42 0.15 True
16.38 ns 42.89 ns 2.62 0.13 True
296.51 ns 1.71 μs 5.78 0.00 True
12.44 μs 14.26 μs 1.15 0.04 False
719.24 ns 1.12 μs 1.56 0.15 False
9.79 ns 21.70 ns 2.22 0.17 True
273.10 ns 1.68 μs 6.16 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph ![graph](<https://pvscmduploaddev.blob.core.windows.net/a

performanceautofiler[bot] commented 7 months ago

utofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Globalization.Tests.StringSearch_15.png>) graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Globalization.Tests.StringSearch* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Globalization.Tests.StringSearch* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, True)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, True)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, True)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, True)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, True)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, True)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False)) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.Sort<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
599.09 μs 669.81 μs 1.12 0.06 False
607.92 μs 692.93 μs 1.14 0.05 False
584.02 μs 650.45 μs 1.11 0.03 False
567.06 μs 703.79 μs 1.24 0.04 True
578.13 μs 698.28 μs 1.21 0.05 True
596.43 μs 669.93 μs 1.12 0.05 False
585.60 μs 649.25 μs 1.11 0.05 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Sort<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Sort<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Sort<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Sort<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.Sort<String>.Array_Comparison(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Sort<String>.Array_ComparerStruct(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Sort<String>.LinqQuery(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Sort<String>.List(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Sort<String>.Array(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Sort<String>.Array_ComparerClass(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Sort<String>.LinqOrderByExtension(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
44.29 ns 280.19 ns 6.33 0.03 True
13.84 ns 79.28 ns 5.73 0.08 True
16.25 ns 96.46 ns 5.93 0.06 True
51.04 ns 196.64 ns 3.85 0.04 True
29.51 ns 465.36 ns 15.77 0.22 True
13.84 ns 59.53 ns 4.30 0.05 True
13.85 ns 73.55 ns 5.31 0.09 True
13.87 ns 79.58 ns 5.74 0.04 True
16.38 ns 96.52 ns 5.89 0.07 True
25.91 ns 59.99 ns 2.32 0.20 True
17.59 ns 199.63 ns 11.35 0.07 True
13.82 ns 69.72 ns 5.04 0.04 True
15.98 ns 133.23 ns 8.34 0.09 True
4.78 ns 105.56 ns 22.07 0.11 True
16.70 ns 232.01 ns 13.89 0.07 True
13.82 ns 73.40 ns 5.31 0.07 True
7.93 ns 107.00 ns 13.49 0.18 True
28.98 ns 464.63 ns 16.03 0.21 True
5.30 ns 53.51 ns 10.10 0.11 True
16.22 ns 132.94 ns 8.19 0.11 True
13.46 ns 113.13 ns 8.41 0.12 True
1.81 ns 94.06 ns 52.00 0.23 True
15.37 ns 68.53 ns 4.46 0.04 True
13.89 ns 55.54 ns 4.00 0.06 True
13.84 ns 76.63 ns 5.54 0.05 True
45.80 ns 87.46 ns 1.91 0.08 True
20.41 ns 147.47 ns 7.23 0.10 True
13.63 ns 55.05 ns 4.04 0.05 True
13.57 ns 62.01 ns 4.57 0.07 True
13.87 ns 71.92 ns 5.19 0.05 True
13.81 ns 68.11 ns 4.93 0.06 True
13.84 ns 79.44 ns 5.74 0.06 True
30.41 ns 63.81 ns 2.10 0.14 True
13.87 ns 55.90 ns 4.03 0.07 True
8.14 ns 107.51 ns 13.21 0.14 True
22.09 ns 204.08 ns 9.24 0.08 True
13.83 ns 80.75 ns 5.84 0.07 True
44.33 ns 116.75 ns 2.63 0.02 True
19.80 ns 122.81 ns 6.20 0.04 True
40.51 ns 155.57 ns 3.84 0.04 True
5.70 ns 69.21 ns 12.14 0.12 True
20.85 ns 147.47 ns 7.07 0.11 True
16.09 ns 50.81 ns 3.16 0.04 True
13.84 ns 78.40 ns 5.66 0.07 True
22.75 ns 100.98 ns 4.44 0.11 True
18.67 ns 144.43 ns 7.74 0.07 True
20.37 ns 147.27 ns 7.23 0.10 True
20.77 ns 147.50 ns 7.10 0.08 True
13.88 ns 58.28 ns 4.20 0.02 True
13.85 ns 77.48 ns 5.59 0.02 True
13.84 ns 86.29 ns 6.23 0.06 True
performanceautofiler[bot] commented 7 months ago

oIl4qtQDdRSM04+A1aFOkIslqIsJWdTOgqepyykSTOcEkYxoSbQsSJojvlecjLyQeRW4XwV96DEpfUw4u6xvvplIaeOfacECWqI7bVZ0l4Ha9oBpZOSpHnRkC1TCX8naip2AdcYq3Jw5WRsyk2LKdQaqZFOqbN5O8YHkdLWQK3ncS/SKeZpaePq2cz78u/tDRwNPkhRRy8jn+L/6V0IAFgv0edFuaq38ILfSJifQg4CxBueY22YmKKeCFrvxH6onUQQvalVTz/I7Xv7MN/6O6FDsmoA13uZNxVbSjZpViCc2XKLl0SFvhh2qDLH2fyw3N2UONtZGtBqu2FS7LwnapmyfXwI2enkk7Q4CxLA6rxoTEVnijKS/Ia5Ladnbz5g96g+FZv382GMf9/sVwfNHrdV6NBqPxef9lr2ejuogy3HY8ewBCcbfmtPTrjM5ESF4EZFM1SoTHKQtdM30xYMAlKlD+sDP6iL0CZXIdoTWe+ieH2eOTO0C6s6mGQUjqXC5z9A4ZiDSvjoi7sDqoTaPdBCTumElGUZN7pjPiK3uDoKKjKCRlInPDRdQ0Ve1xFe1dWKJmOofIe2tvzz4t1SF0EuILFThMFTnTES1BBb8BCzayAv8EAAA=>) | 43.80 ns | 155.53 ns | 3.55 | 0.05 | True | | |

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Matrix4x4* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Matrix4x4* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateReflectionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromVectorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.EqualityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.InvertBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.IsIdentityBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateFromMatrix3x2 #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveFieldOfViewBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.SubtractBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveOffCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.IdentityBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.InequalityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.GetDeterminantBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.SubtractOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.Transpose #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateFromAxisAngleBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.00 ns 284.23 ns ∞ 0.00 True
582.68 ns 1.10 μs 1.89 0.00 True
581.66 ns 1.09 μs 1.87 0.00 True
0.00 ns 321.81 ns ∞ 0.00 True
280.28 ns 510.55 ns 1.82 0.02 True
198.96 ns 241.14 ns 1.21 0.01 True
6.81 μs 9.04 μs 1.33 0.01 False
7.90 μs 11.73 μs 1.49 0.01 True
0.00 ns 221.31 μs ∞ 0.00 True
110.34 μs 221.61 μs 2.01 0.00 True
71.31 μs 105.04 μs 1.47 0.02 False
0.00 ns 221.39 μs ∞ 0.00 True
293.63 ns 312.27 ns 1.06 0.02 False
6.47 μs 8.94 μs 1.38 0.01 False
213.77 ns 303.89 ns 1.42 0.01 True
45.21 μs 55.16 μs 1.22 0.02 True
0.00 ns 476.86 ns ∞ 0.00 True
8.34 μs 12.22 μs 1.47 0.02 True
1.64 μs 2.96 μs 1.80 0.02 True
118.60 μs 246.15 μs 2.08 0.00 True
1.61 μs 2.36 μs 1.47 0.01 True
71.31 μs 104.67 μs 1.47 0.01 False
356.49 ns 719.81 ns 2.02 0.02 True
232.30 ns 259.69 ns 1.12 0.03 True
0.00 ns 480.67 ns ∞ 0.00 True
70.41 μs 103.50 μs 1.47 0.01 False
0.00 ns 260.99 ns ∞ 0.01 True
0.00 ns 189.56 μs ∞ 0.00 True
1.97 μs 2.93 μs 1.48 0.04 False
213.78 ns 304.01 ns 1.42 0.00 True
60.18 μs 73.81 μs 1.23 0.01 True
386.46 ns 832.09 ns 2.15 0.01 True
173.74 ns 322.77 ns 1.86 0.00 True
0.00 ns 262.00 ns ∞ 0.00 True
174.13 ns 228.37 ns 1.31 0.01 True
1.05 μs 1.67 μs 1.59 0.03 True
575.08 ns 1.10 μs 1.91 0.02 True
295.54 ns 508.85 ns 1.72 0.01 True
174.17 ns 225.12 ns 1.29 0.00 True
0.00 ns 188.44 μs ∞ 0.00 True
227.50 ns 275.01 ns 1.21 0.02 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link:

performanceautofiler[bot] commented 7 months ago

https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.RegularExpressions.Tests.Perf_Regex_Common* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.RegularExpressions.Tests.Perf_Regex_Common* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in GuardedDevirtualization.ThreeClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.81 ns 5.11 ns 1.34 0.05 False
3.33 ns 4.89 ns 1.47 0.04 False
3.58 ns 4.89 ns 1.36 0.05 False
3.39 ns 5.12 ns 1.51 0.04 False
3.33 ns 4.83 ns 1.45 0.05 False
3.37 ns 5.13 ns 1.52 0.04 False
4.98 ns 7.10 ns 1.43 0.04 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'GuardedDevirtualization.ThreeClassInterface*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter GuardedDevirtualization.ThreeClassInterface* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'GuardedDevirtualization.ThreeClassInterface*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter GuardedDevirtualization.ThreeClassInterface* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.67) #### ETL Files #### Histogram #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.00) #### ETL Files #### Histogram #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.33) #### ETL Files #### Histogram #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.00) #### ETL Files #### Histogram #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.67) #### ETL Files #### Histogram #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.33) #### ETL Files #### Histogram #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.33) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
41.88 ns 347.83 ns 8.30 0.02 True
52.60 ns 6.28 μs 119.40 0.13 True
87.05 ns 1.57 μs 17.98 0.01 True
57.36 ns 315.64 ns 5.50 0.01 True
81.55 ns 392.28 ns 4.81 0.01 True
119.54 ns 1.60 μs 13.42 0.01 True
12.81 ns 499.29 ns 38.96 0.14 True
15.56 ns 127.97 ns 8.22 0.16 True
56.63 ns 299.55 ns 5.29 0.03 True
85.46 ns 6.10 μs 71.39 0.02 True
14.97 ns 137.11 ns 9.16 0.22 True
63.26 ns 1.58 μs 25.01 0.03 True
49.64 ns 130.88 ns 2.64 0.04 True
88.52 ns 1.57 μs 17.72 0.02 True
146.13 ns 1.72 μs 11.80 0.02 True
12.92 ns 451.83 ns 34.97 0.09 True
52.79 ns 6.25 μs 118.33 0.13 True
107.74 ns 6.19 μs 57.45 0.01 True
68.27 ns 6.16 μs 90.17 0.01 True
61.83 ns 1.60 μs 25.91 0.04 True
108.21 ns 208.55 ns 1.93 0.05 True
85.93 ns 6.09 μs 70.83 0.01 True
82.84 ns 199.44 ns 2.41 0.03 True
48.72 ns 130.90 ns 2.69 0.02 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Encodings.Web.Tests.Perf_Encoders* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Encodings.Web.Tests.Perf_Encoders* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+<World>!,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,�2020,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,�2020,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,512) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+<World>!,16) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",16) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.92 ns 21.18 ns 3.57 0.36 True
10.99 ns 157.57 ns 14.33 0.09 True
44.83 ns 60.77 ns 1.36 0.05 True
454.71 ns 1.54 μs 3.39 0.03 True
10.06 ns 147.48 ns 14.66 0.04 True
457.50 ns 3.42 μs 7.47 0.02 True
10.15 ns 35.14 ns 3.46 0.01 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Buffers.Binary.Tests.BinaryReadAndWriteTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Buffers.Binary.Tests.BinaryReadAndWriteTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseLE #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseEndianness #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseUsingNtoH #### ETL Files #### Histogram #### JIT Disasms ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
55.93 ns 64.35 ns 1.15 0.07 False
257.96 ns 381.25 ns 1.48 0.04 False
238.83 ns 307.90 ns 1.29 0.07 True
3.83 μs 7.17 μs 1.87 0.09 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Diagnostics.Perf_Activity*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Diagnostics.Perf_Activity* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Diagnostics.Perf_Activity*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Diagnostics.Perf_Activity* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Diagnostics.Perf_Activity.EnumerateActivityTagsSmall #### ETL Files #### Histogram #### JIT Disasms ### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C) #### ETL Files #### Histogram #### JIT Disasms ### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: Hierarchical) #### ETL Files #### Histogram #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityTagsLarge #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
41.10 ns 85.19 ns 2.07 0.05 False
1.49 μs 2.93 μs 1.97 0.03 True
1.27 ms 2.42 ms 1.91 0.02 True
713.47 ns 1.25 μs 1.75 0.01 True
16.93 ns 35.91 ns 2.12 0.09 False
717.68 ns 1.20 μs 1.68 0.01 True
5.07 ns 18.45 ns 3.64 0.14 True
14.50 ns 19.28 ns 1.33 0.10 True
14.07 ns 24.78 ns 1.76 0.05 True
108.20 ns 189.10 ns 1.75 0.02 True
11.55 ns 27.17 ns 2.35 0.12 True
14.87 ns 18.74 ns 1.26 0.07 True
422.52 ns 702.65 ns 1.66 0.02 True
112.31 ns 157.37 ns 1.40 0.00 True
14.37 ns 18.85 ns 1.31 0.08 True
5.14 ns 18.44 ns 3.59 0.13 True
841.14 μs 1.61 ms 1.91 0.02 False
7.70 μs 11.17 μs 1.45 0.02 True
168.57 ns 222.17 ns 1.32 0.04 False
23.17 ns 28.27 ns 1.22 0.06 False
1.62 ms 5.81 ms 3.58 0.02 True
8.58 ns 22.32 ns 2.60 0.16 True
16.72 ns 22.32 ns 1.34 0.04 True
7.84 ns 22.62 ns 2.89 0.15 True
19.57 ns 25.39 ns 1.30 0.03 False
7.72 μs 9.55 μs 1.24 0.03 False
105.12 ns 118.85 ns 1.13 0.01 False
62.32 ns 102.47 ns 1.64 0.05 True
90.09 ns 125.83 ns 1.40 0.04 True
109.43 μs 221.08 μs 2.02 0.03 True
5.55 ms 9.96 ms 1.79 0.02 True
14.43 ns 18.76 ns 1.30 0.10 False
15.02 ns 20.75 ns 1.38 0.02 True
788.91 ns 1.66 μs 2.10 0.04 False
87.15 ns 185.52 ns 2.13 0.04 True
11.27 ns 19.69 ns 1.75 0.07 True
6.53 ns 20.62 ns 3.16 0.19 True
76.67 ns 88.43 ns 1.15 0.00 True
1.95 ms 3.79 ms 1.95 0.01 True
75.10 ns 114.39 ns 1.52 0.06 True
18.02 ns 35.45 ns 1.97 0.10 False
1.63 ms 5.65 ms 3.47 0.02 True
51.53 ns 73.58 ns 1.43 0.07 True
680.02 ns 3.80 μs 5.58 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph ![graph](<https://pvscmduploaddev.blob.core

performanceautofiler[bot] commented 7 months ago

.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Numerics.Tests.Perf_BigInteger_33.png>) graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_BigInteger* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_BigInteger* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 1024,1024 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 65536,65536 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 1024,512 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 259 bytes, Same) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 1024,512 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, DiffFirstByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 16,16 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: -2147483648) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 259 bytes, DiffMiddleByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 16,16,16 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 16,8 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 259 bytes, DiffFirstByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 65536,32768 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 1024,1024 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: -2147483648) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 65536,32768 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, DiffLastByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, Same) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 123) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Add(arguments: 65536,65536 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: -2147483648) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 1024,1024,64 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 65536,65536 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Add(arguments: 16,16 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 16,8 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 1024,512 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Add(arguments: 1024,1024 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 67 bytes, DiffMiddleByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 16,16 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 16384,16384,64 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: -2147483648) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Equals(arguments: 259 bytes, DiffLastByte) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123) #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
533.56 ns 864.40 ns 1.62 0.05 True
38.13 ns 46.35 ns 1.22 0.06 False
42.68 ns 825.60 ns 19.35 0.01 True
202.67 ns 333.19 ns 1.64 0.01 True
309.80 ns 5.31 μs 17.13 0.02 True
4.17 ns 5.36 ns 1.29 0.18 False
321.94 ns 526.56 ns 1.64 0.04 True
437.59 ns 756.71 ns 1.73 0.04 True
565.35 ns 5.77 μs 10.21 0.03 True
106.45 ns 150.23 ns 1.41 0.01 True
114.17 ns 151.96 ns 1.33 0.03 True
89.74 ns 252.02 ns 2.81 0.10 True

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Perf_Convert*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Perf_Convert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Perf_Convert*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Perf_Convert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009") #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ChangeType #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.FromHexString #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ToDateTime_String(value: "12/12/1999") #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.GetTypeCode #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009") #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM") #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ToBase64String(formattingOptions: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.FromBase64String #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.FromBase64Chars #### ETL Files #### Histogram #### JIT Disasms ### System.Perf_Convert.ToHexString #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
235.69 ns 311.11 ns 1.32 0.09 False
1.12 μs 1.86 μs 1.66 0.04 True
6.23 μs 10.01 μs 1.61 0.01 True
4.77 μs 10.51 μs 2.20 0.01 True
1.06 μs 1.73 μs 1.64 0.05 True
7.03 μs 15.57 μs 2.21 0.05 True
1.07 μs 1.70 μs 1.59 0.12 True
1.06 μs 1.64 μs 1.54 0.10 True
5.87 μs 9.87 μs 1.68 0.06 True
5.74 μs 23.84 μs 4.16 0.04 True
939.68 ns 1.76 μs 1.87 0.00 True
5.04 μs 8.92 μs 1.77 0.02 True
6.40 μs 16.18 μs 2.53 0.02 True
1.05 μs 1.68 μs 1.60 0.09 True
1.23 μs 1.75 μs 1.42 0.09 False
1.14 μs 1.85 μs 1.62 0.10 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Perf_Get* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Perf_Get* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Tests.Perf_Get.GetBoolean #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetInt16 #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetSingle #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetDateTime #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetUInt16 #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetInt64 #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetUInt64 #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetDecimal #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetString #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetByte #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetDouble #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetGuid #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetUInt32 #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetInt32 #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Get.GetSByte #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.01 μs 2.33 μs 1.16 0.01 True
400.01 ns 513.31 ns 1.28 0.03 True
384.46 ns 496.90 ns 1.29 0.02 True
7.31 ms 23.12 ms 3.16 0.01 True
3.24 μs 5.40 μs 1.67 0.01 True
3.43 μs 3.67 μs 1.07 0.02 False
501.24 ns 810.52 ns 1.62 0.06 True
2.96 μs 5.02 μs 1.70 0.01 True
5.47 μs 9.94 μs 1.82 0.02 True
5.31 μs 9.57 μs 1.80 0.02 True
7.52 ms 23.46 ms 3.12 0.01 True
2.23 μs 2.68 μs 1.20 0.03 True
553.75 ns 836.16 ns 1.51 0.05 True

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_DocumentParse* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_DocumentParse* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: BasicJson) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
239.76 μs 485.61 μs 2.03 0.03 True
177.12 μs 405.42 μs 2.29 0.01 True
12.34 μs 72.16 μs 5.85 0.09 True
204.70 μs 352.54 μs 1.72 0.03 True
352.02 μs 671.18 μs 1.91 0.03 True
7.77 μs 152.38 μs 19.62 0.01 True
95.78 μs 538.53 μs 5.62 0.01 True
309.33 μs 1.13 ms 3.66 0.02 True
286.05 μs 590.87 μs 2.07 0.00 True
184.31 μs 718.86 μs 3.90 0.02 True
13.42 μs 78.94 μs 5.88 0.09 True
148.87 μs 254.67 μs 1.71 0.01 True
27.32 μs 157.40 μs 5.76 0.09 True
215.15 μs 355.98 μs 1.65 0.01 True
10.66 μs 62.88 μs 5.90 0.09 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Perf_Utf8Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Perf_Utf8Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Greek) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Greek) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.84 ns 19.69 ns 5.13 0.10 True
4.43 ns 36.48 ns 8.23 0.08 True
3.56 ns 9.87 ns 2.77 0.12 True
3.50 ns 11.77 ns 3.36 0.08 True
3.60 ns 9.89 ns 2.75 0.11 True
0.53 ns 1.99 ns 3.76 0.14 True
3.77 ns 9.91 ns 2.63 0.10 True
3.77 ns 9.92 ns 2.63 0.14 True
3.29 ns 8.41 ns 2.55 0.09 True
3.60 ns 9.95 ns 2.76 0.06 True
3.15 ns 6.89 ns 2.19 0.08 True
3.51 ns 14.29 ns 4.07 0.08 True
3.49 ns 11.76 ns 3.37 0.09 True
0.65 ns 7.17 ns 11.04 0.38 True
3.68 ns 9.92 ns 2.70 0.06 True
4.21 ns 32.22 ns 7.66 0.10 True
3.05 ns 8.26 ns 2.71 0.07 True
0.47 ns 2.51 ns 5.31 0.19 True
3.44 ns 9.95 ns 2.89 0.09 True
3.77 ns 9.97 ns 2.64 0.15 True
3.60 ns 11.28 ns 3.13 0.11 True
3.88 ns 22.32 ns 5.75 0.10 True
3.29 ns 18.82 ns 5.71 0.11 True
3.52 ns 9.91 ns 2.82 0.11 True
3.76 ns 25.01 ns 6.64 0.07 True
3.55 ns 11.95 ns 3.37 0.15 True
0.34 ns 6.39 ns 18.90 0.32 True
3.26 ns 8.41 ns 2.58 0.11 True
2.61 ns 8.24 ns 3.16 0.09 True
0.32 ns 2.19 ns 6.79 0.44 True
3.31 ns 8.29 ns 2.51 0.15 True
4.29 ns 93.63 ns 21.81 0.09 True
1.18 ns 11.52 ns 9.77 0.25 True
3.44 ns 10.98 ns 3.19 0.10 True
0.33 ns 23.10 ns 70.25 0.55 True
3.69 ns 20.34 ns 5.52 0.10 True
4.54 ns 101.01 ns 22.26 0.10 True
0.28 ns 2.62 ns 9.37 0.46 True
3.41 ns 16.25 ns 4.76 0.11 True
3.19 ns 9.95 ns 3.12 0.13 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\perform
performanceautofiler[bot] commented 7 months ago

ance

One line run:

python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

Individual Commands:

Restore

dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

Build

dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

Run

dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200


</details>

<details>

### Payloads

[Baseline](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-9cc7262c-4aa6-4161-a055-c5c8c938adde145147d624d4820aa/bf4af9fa-c2c5-42cc-9e42-d2d27a31bb2e.zip?sv=2021-08-06&se=2024-03-20T17%3A33%3A41Z&sr=c&sp=rl&sig=2Gqka3CMZaJP4Pu18xolXyflqKHEHvSHSLflljK80d0%3D>)
[Compare](<
                    https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>)
### System.Numerics.Tests.Perf_Vector2.LerpBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DivideByVector2OperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.MultiplyByScalarOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.SubtractOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.EqualityOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.MultiplyOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DivideByVector2Benchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.CreateFromScalar

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.SubtractFunctionBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.ZeroBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.AbsBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DivideByScalarOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.LengthBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.NegateBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.UnitXBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.InequalityOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.AddOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.MultiplyFunctionBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.MaxBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.ReflectBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.SquareRootBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.AddFunctionBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.TransformByQuaternionBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DivideByScalarBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DistanceSquaredBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.OneBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.UnitYBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DotBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.CreateFromScalarXYBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.DistanceBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.MinBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.EqualsBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.NormalizeBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.LengthSquaredBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.ClampBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Numerics.Tests.Perf_Vector2.NegateOperatorBenchmark

#### ETL Files

#### Histogram

#### JIT Disasms

### Docs

[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

</details>

---

### Run Information

Name | Value
-- | --
Architecture | x64
OS | ubuntu 22.04
Queue | TigerUbuntu
Baseline | [5e1caf14fd94b4ed85357002303c8935fedacb8b](https://github.com/dotnet/runtime/commit/5e1caf14fd94b4ed85357002303c8935fedacb8b)
Compare | [9dc6ea62a4d195ae4559f4609a56933c61889756](https://github.com/dotnet/runtime/commit/9dc6ea62a4d195ae4559f4609a56933c61889756)
Diff | [Diff](https://github.com/dotnet/runtime/compare/5e1caf14fd94b4ed85357002303c8935fedacb8b...9dc6ea62a4d195ae4559f4609a56933c61889756)
Configs | CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

### Regressions in StoreBlock.LocalAddress

Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio
-- | -- | -- | -- | -- | -- | -- | -- | --
|<ul><li>[InitBlockAllOnes16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllOnes16.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlmbpl01VQoiLQMKLRtdxA1Ck5ucNgZ/RMfO1kyI346TllJl0gg3kf3mec85/jheAtUlggBp9PNnP8l9DghkWjKeraBQJIqIkykjwfSxlIYJcFrYFKlM8wZE2Oh+DjTTfUGZbJMxpjkzkBqbkDBJXGc3Hjk+cSgKO/D+4glo84kKqKO+dm6MQphylf7oLVRKeZxlCFr35pKZRo45v5KgB+OTjKtSzpTcsG2J1DAl9VdnpkTBeDNbqgycbyT6RRzDLJ497Vwsviz/4FjC0/BSSRVfJf/Fz6UBLBDs9+DbUK7/YbTSRyazg0GwFNWtsMFOXEyDKEzlPnLHyngdsLnSC34nmuPrwn/Ql3LLJHRhr7cqqYpuaF5pZk/+GpVQtdLV+L5aI8vecbWmnD00WhfbKlx1zXC5KwBtW8jmwrbu8EzZngFMVHEYtToiscYbQ0VBXpGMGqj7yw2DcHg2GJyFF8lgMBmNJufnveAiHA/D8csgmARBk0SXQlBkD0Aobm8F3blNxLoIn7zwyLpqpfCPXeXXl+lzCSXUgQpU321PPqJXoEtuIrSFZ+7JYvZ6fAdItzbU0PNJE6uOHL1FBjLj1VGpN6yZNEWj/QlI6mWmOUVD7pnJiavtDoKOjiaf7FLFSyGjdlHVXtfRvgoLGmY4RM4bu3v2KakO09pCXKm9WtMFZyaiO9Deb65xbZrvBAAA>)</li></ul> | 4.01 ns | 6.04 ns | 1.51 | 0.07 | False | | |
|<ul><li>[InitBlockAllOnes128 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllOnes128.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlk/snYqlYJoxxiFlo0u4gahyY1PG4M/omNnaybEb8dJS6kyaYSbyH7zvOccfxwvgJoCQYKy5vmzn+Q+AwQyLbhgS8g1iWPiMW0V2C4WynIJXgObIlVpVoMIa9PNgDLTlZSrJjnBNOMWUusSEq6I723PBl5IPIrSDYK/eALGfqISqqhvvBurEaZCpz86c51SMWEMwZjOTHFbyxMhrhSYfjQ6Srks1LlWa74pkFqulfnqnWuZc1HPFpqB943Ev4hnucPZ0875/MviD44FPA0vtNKTq+S/+JmygDmC++59ayrMP4xO+sgV2xskT1HfShfsyMUNyNyW/iP3RNugBTbTZi7uZH1+bfgP5kJtuII27PVGJ2XeDs1Kw93RX6OWulLaGt+XK+TsUugVFfyh1trYltGybYaLbQ7o+kLVN7Zxic+1axrAROf7UaMlEme8sVTm5DVh1ELVYH7Ui05P+v2TaJT0++PBYDwcdqLh6NVZFL3s9ca9Xp3EFFJS5A9AKG5uJd36dcSqiJC8CMiqbKQID20VVpfpcwEFVIFy1N9dUz6il2AKYWN0hTP/aDE7fXIHSDcu1GkQkjpWFTl+hxwUE+VBqTasntRFo/sJSKplphlFS+65zYhv3A6CiQ+mkGxTLQqp4mZR5U438a4KB1puBcTeW7d77i0p99PKQnxlgkozueA2plswwW/c/QuK8AQAAA==>)</li></ul> | 8.49 ns | 16.35 ns | 1.93 | 0.10 | True | | |
|<ul><li>[InitBlockAllOnes64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllOnes64.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlm/1qJRKYi0jFFo2egibhCa3Pi0MfgjOna2Zpr223HS0k2ZNMJNZL953tfHjk8WQE2BIEFZ8/LFHbnJAIFMCi7YEnJNooh4TFsFtouFslyC18AmSFWa1SDC2nQzoMx0JeWqScaYZtxCat2ChCvie9s3Qy8kHkXpBsEDnoCxX6mEKvW9d2k1wkTo9HdnrlMqYsYQjOnMFLe1HAtxrsC4jIeIZaGmWq35pkBquVbmhzfVMueini00A+8nie6JZ7nD2fPO+fz74i+OBTwPL7TS8XnyX/xMWcAcwT33vjUV5h9GJ33hiu0Nkqeor6QLe+TiBmRuS/+JO9Y2aIHNtJmLa1l/vjb8Z3OqNlxBG/Zio5Myb4dmpeHuy1+glrpS2ho/lSvk7EzoFRX8ttba2JaDZdsVTrc5oGsLVV/Yxh2eatczgInO96NGRyTOeGmpzMk7wqiFqr/8QW9wfNTvHw1Okn5/PByOR6POcDDqn7wdve71xr1evYgppKTIb4FQ3FxJuvXrxKqIkLwKyKpsLBEeuiqsLtO3AgqognLUv1xPPqGXYAphI3SFM//RZnZ6fA1INy7qOAhJnVUlRx+Rg2KiPCjVgdWTumh0LwFJtc00o2jJDbcZ8Y07QTDRwRSSbapFIVXULKrc6SbaVeFAy62AyPvgTs/9Ssr9tLIQX5mg0kwuuI3oFkzwB7YxmCzvBAAA>)</li></ul> | 5.97 ns | 9.81 ns | 1.64 | 0.10 | True | | |
|<ul><li>[InitBlockAllOnes32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllOnes32.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlm/i9ZKQbRljELLRhdxg9DkxqeNwR/RsbM1E+K346SlVJk0wk1kv3nec44/jpdATY4gQVnz/NlPcp8CApnmXLAVZJpEEfGYtgpsG3NluQSvhk2RqiStQISNaadAmWlLylWdnGCScguJdQkJV8T3dq8GXkg8itINgr94DMZ+ohLKqG+8G6sRpkInP1oLnVAxYQzBmNZccVvJEyGuFJh+7yTjKlczrTZ8myO1XCvz1ZtpmXFRzZaagfeNRL+IZ7nD2dPOxeLL8g+OOTwNL7XSk6v4v/i5soAZgvsefBsqzD+MTvrIFTsYJE9Q30oX7MTFDcjMFv4j90TboAE212Yh7mR1fE34D+ZCbbmCJuz1VsdF1gxNC8PdyV+jlrpUmhrfF2vk7FLoNRX8odKa2Fa9VdMMF7sM0LWFqi5s7Q7PtOsZwFhnh1GtI2JnvLFUZuQ1YdRC2V9+r9Prn3W7Z73zuNsdDwbj4bA1PO+PRqPuy05n3OlUSUwuJUX+AITi9lbSnV9FLIsIyYuArItaivDYVWF5mT7nkEMZKEP93fXkI3oFJhc2Qlc4808Ws9cnd4B060L1g5BUscrI0TvkoJgojkq5YdWkKhrdT0BSLjNJKVpyz21KfON2EEx0NIVkl2iRSxXViyr2uon2VTjQcisg8t663XNPSXGYlhbiKxOUmskEtxHdgQl+A+s1JhXvBAAA>)</li></ul> | 5.20 ns | 7.12 ns | 1.37 | 0.11 | False | | |
|<ul><li>[CopyBlock128 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.CopyBlock128.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlm/lpZRKYi2bDBoYXQRNwhNbnzaGPwRHTtbMyF+O05aSpVJI9xE9pvnPef443gB1BQIEpQ1T5/8JHcZIJBpwQVbQq5JHBOPaavAdrFQlkvwGtgUqUqzGkRYm24GlJmupFw1yQmmGbeQWpeQcEV8bzuKvJB4FKUbBH/xBIz9SCVUUV9711YjTIVOf3TmOqViwhiCMZ2Zzsta7g/OjnItCzXTas03BVLLtTJfvZmWORf1bKEZeN9I/It4ljucPe6cz78s/uBYwOPwQis9+ZT8F3+pLGCO4L5735oK8w+jkz5wxfYGyVPUN9IFO3JxAzK3pf/APdE2aIFdajMXt7I+uDb8e3OuNlxBG/Zqo5Myb4dmpeHuzK9QS10pbY3vyhVy9lboFRX8vtba2JaDZdsM59sc0DWEqq9q4/bOtOsWwETn+1GjFxJnvLZU5uQVYdRC1Vn+oDc4Pen3TwZnSb8/jqLxcNh5EY1eRqPh815v3OvVSUwhJUV+D4Ti5kbSrV9HrIoIybOArMpGivDQT2F1mT4XUEAVKEf93XXjA3oJphA2Rlc4848Ws9Mnt4B040KdBiGpY1WR4wvkoJgoD0q1YfWkLhrdT0BSLTPNKFpyx21GfON2EEx8MIVkm2pRSBU3iyp3uol3VTjQcisg9t643XOPSLmfVhbiKxNUmskFtzHdggl+Ay6qnhPpBAAA>)</li></ul> | 0.00 ns | 5.09 ns | 465583.63 | 0.62 | True | | |
|<ul><li>[InitBlockAllZeros128 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllZeros128.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAl2bdi0dk4JIy4BCC6OLuAChyY1PG4M/omNnaybEb8dJS6kyaYSbyH7zvOccfxwvgJoCQYKy5vGjn+Q2AwQyKbhgS8g1iSLiMW0V2B4WynIJXgObIFVpVoMIa9PLgDLTk5SrJhljmnELqXUJCVfE97bPhl6HeBSlGwR/8QSM/UAlVFFfeldWI0yETn905zqlImYMwZjuTHFby7EQXwC16Q/OjnIuCzXVas03BVLLtTJfvamWORf1bKEZeN9I9It4ljucPeyczz8v/uBYwMPwQisdf0z+i58pC5gjuO/et6bC/MPopPdcsb1B8hT1tXTBjlzcgMxt6d9zx9oGLbCZNnNxI+sDbMO/MxdqwxW0YS83OinzdmhWGu7O/hK11JXS1vi2XCFnb4ReUcHvaq2NbTlYts1wsc0BXWOo+so2bvFUu64BTHS+HzV6InHGK0tlTl4QRi1UHeYPwsHpSb9/MjhL+v3z4fB8NOo+D8fhaDx+GobnYVgnMYWUFPkdEIqba0m3fh2xKqJDngRkVTZSdA591aku06cCCqgC5ai/u668Ry/BFMJG6Apn/tFidnp8A0g3LtRp0CF1rCpy9Bo5KCbKg1JtWD2pi0b3E5BUy0wzipbccpsR37gdBBMdTB2yTbUopIqaRZU73US7KhxouRUQea/c7rnHpNxPKwvxlQkqzeSC24huwQS/Af9UKubxBAAA>)</li></ul> | 0.00 ns | 4.50 ns | 1350.01 | 0.22 | True | | |
|<ul><li>[InitBlockAllZeros64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllZeros64.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlmbfjCNSkGkY4xCC6OLuAChyY1PG4M/omNnaybEb8dJSzdl0gg3kf3meV8fOz5ZADUlggRlzdMnv8hNDghkWnLBllBoEsfEY9oqsH0sleUSvBY2RaqyvAER1qafA2WmLylXbTLBLOcWMusWJFwR39sej72QeBSlGwR3eArGfqQS6tTX3qXVCFOhs5+9uc6oSBhDMKY3U9w2ciLEV0BtXMhdxrJUp1qt+aZEarlW5pt3qmXBRTNbaAbedxL/Jp7lDmePO+fzL4u/OJbwOLzQSief0v/iZ8oCFgjuufetqTD/MDrpA1dsb5A8Q30lXdg9FzcgC1v5D9yJtkEHbKbNXFzL5vt14d+bM7XhCrqwFxudVkU3NK8Md5/+ArXUtdLV+K5aIWfnQq+o4LeN1sW2HC67rnC2LQBdX6jmxrYu8al2TQOY6mI/arVE6oyXlsqCvCKMWqgbzB9Gw9HRYHA0PEkHg8l4PHlx3ItORi+jcfQ8iiZR1CxiSikp8lsgFDdXkm79JrEuIiTPArKqWkuEh7YK68v0uYQS6qAC9Q/XlA/oJZhS2Bhd4cy/t5mdnlwD0o2LGgUhabLq5PgtclBMVAelPrBm0hSN7iUgqbeZ5RQtueE2J75xJwgmPphCss20KKWK20VVO93EuyocaLkVEHtv3Om5f0m1n9YW4isT1JopBLcx3YIJ/gDEUCOd8AQAAA==>)</li></ul> | 0.00 ns | 2.23 ns | 1040.62 | 0.65 | True | | |
|<ul><li>[CopyBlock32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.CopyBlock32.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWU247TMBCG75F4B98lgWwPaSmoUhBtWWChhSUbcYPQyo2njcGHaOzsNivEs+OkpVRZaQk3kf3n+2fGh/EKqCkRJChrHj/6SW5zQCDzkguWQKFJHBOPaavA9rFUlkvwWtgcqcryBkTYmH4OlJm+pFy1yRlmObeQWZeQcEV8bzcZeyHxKEo3CP7iKRj7kUqoo77yrqxGmAud/egtdUbFjDEEY3oLXVSNPIpOUiWlWmi14dsSqeVama/eQsuCi2a20gy8byT+RTzLHc4edi6XX1Z/cCzhYXillZ59Sv+Lv1AWsEBw34NvQ4X5h9FJH7hiB4PkGepr6YKduLgBWdjKv+eeaRt0wC60WYob2ZxbF/69OVdbrqALe7nVaVV0Q/PKcHfkl6ilrpWuxnfVGjl7K/SaCn7XaF1sSZR0zXC+KwBdP6jmprYu70K7ZgFMdXEYtVohdcYrS2VBXhJGLdSN5UeDaHQ2HJ5FL9LhcDoeT59NetF4PBk9j54OBtPBoEliSikp8jsgFLfXku78JmJdREieBGRdtVKEx3YK68v0uYQS6kAF6u+uGe/RCZhS2Bhd4cw/Wcxen90A0q0LNQpC0sSqI8dvkINiojoq9YY1k6ZodD8BSb3MLKdoyS23OfGN20Ew8dEUkl2mRSlV3C6q2usm3lfhQMutgNh77XbPvSHVYVpbiK9MUGumENzGdAcm+A0imxxx6AQAAA==>)</li></ul> | 0.00 ns | 1.32 ns | 685.10 | 0.56 | True | | |
|<ul><li>[InitBlockAllZeros32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllZeros32.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlm/V2BSEGnZoNDC6CIuQGhy49PG4I/o2NmaCfHbcdJSqkwa4Say3zzvOccfxwugpkCQoKx5/Ognuc0AgUwKLtgSck2iiHhMWwW2i4WyXILXwCZIVZrVIMLadDOgzHQl5apJxphm3EJqXULCFfG97XjkhcSjKN0g+IsnYOwHKqGK+sq7shphInT6ozPXKRUxYwjGdGaK21qOhfgCqM1wcJRyWaipVmu+KZBarpX56k21zLmoZwvNwPtGol/Es9zh7GHnfP558QfHAh6GF1rp+GPyX/xMWcAcwX33vjUV5h9GJ73niu0Nkqeor6ULduTiBmRuS/+eO9Y2aIHNtJmLG1mfXxv+nTlXG66gDXu50UmZt0Oz0nB39Jeopa6Utsa35Qo5eyP0igp+V2ttbMvBsm2G820O6PpC1Te2cYmn2jUNYKLz/ajREokzXlkqc/KSMGqhajB/0BsMT/r9k8HzpN8/G43OTsedUf90/Gz04mmvd9br1UlMISVFfgeE4uZa0q1fR6yKCMmTgKzKRorw0FZhdZk+FVBAFShH/d015T16CaYQNkJXOPOPFrPT4xtAunGhhkFI6lhV5OgCOSgmyoNSbVg9qYtG9xOQVMtMM4qW3HKbEd+4HQQTHUwh2aZaFFJFzaLKnW6iXRUOtNwKiLzXbvfcW1Lup5WF+MoElWZywW1Et2CC34B22bLwBAAA>)</li></ul> | 0.00 ns | 1.14 ns | 534.59 | 0.40 | True | | |
|<ul><li>[CopyBlock64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.CopyBlock64.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlk/0m6MSkG0ZcCghdFF3CA0ufFpY/BHdOxsyYT47Thp6aZMGuEmst887znHH8dLoKZAkKCsefrkF7nJAIHMCi7YCnJN4ph4TFsFto+FslyC18JmSFWaNSDCxvQzoMz0JeWqTU4xzbiF1LqEhCvie+XJ2AuJR1G6QXCHJ2DsJyqhjvrau7QaYSZ0+rO30CkVU8YQjOnNdV41sjPfeVeFmmu14dsCqeVamW/eXMuci2a21Ay87yT+TTzLHc4edy4WX5d/cSzgcXiplZ5+Tv6LP1cWMEdw371vQ4X5h9FJH7lie4PkKeor6YLdc3EDMreV/8A91TbogJ1rsxDXsjm3LvwHc6a2XEEX9mKrkyrvhmaV4e7IL1BLXStdje+rNXL2Tug1Ffy20brYVtGqa4azMgd0/aCam9q6vHPtmgUw0fl+1GqFxBkvLZU5eUUYtVA3lh8NotHRcHgUnSbD4WQ8nhyf9I5fjE5fRtHzwWAyGDRJTCElRX4LhOL2StLSbyLWRYTkWUDWVStFeGinsL5MXwoooA6Uo/7hmvEBvQJTCBujK5z59xaz06fXgHTrQo2CkDSx6sjxW+SgmKgOSr1hzaQpGt1PQFIvM80oWnLDbUZ843YQTHwwhaRMtSikittFVTvdxLsqHGi5FRB7b9zuuTek2k9rC/GVCWrN5ILbmJZggj8edxce6AQAAA==>)</li></ul> | 0.00 ns | 2.53 ns | 2614.14 | 0.36 | True | | |
|<ul><li>[InitBlockAllOnes8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/StoreBlock.LocalAddress.InitBlockAllOnes8.html>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWUXW/TMBSG75H4D75LAlm/KaVSEG3ZoNCy0UXcIDS58Wlj8Ed07GzNhPjtOGkpVSaNcBPZb573nOOP4yVQkyNIUNY8ffKT3KWAQKY5F2wFmSZRRDymrQLbxlxZLsGrYVOkKkkrEGFj2ilQZtqSclUnJ5ik3EJiXULCFfG93XDghcSjKN0g+IvHYOwnKqGM+sa7thphKnTyo7XQCRUTxhCMac0Vt5U8EeJSgRmdJFzlaqbVhm9zpJZrZb56My0zLqrZUjPwvpHoF/Esdzh73LlYfFn+wTGHx+GlVnpyGf8XP1cWMENw34NvQ4X5h9FJH7liB4PkCeob6YKduLgBmdnCf+CeaBs0wObaLMStrE6vCf/BnKstV9CEvdrquMiaoWlhuDv4K9RSl0pT4/tijZy9E3pNBb+vtCa2VW/VNMP5LgN0XaGq+1q7wjPtWgYw1tlhVGuI2BmvLZUZeU0YtVC2l9/r9Ppn3e5ZbxR3u+PBYPxi2HrZe9UfjgbPO51xp1MlMbmUFPk9EIrbG0l3fhWxLCIkzwKyLmopwmNTheVl+pxDDmWgDPV315IP6BWYXNgIXeHMP1nMXp/cAtKtC9UPQlLFKiNHF8hBMVEclXLDqklVNLqfgKRcZpJStOSO25T4xu0gmOhoCsku0SKXKqoXVex1E+2rcKDlVkDkvXW7516S4jAtLcRXJig1kwluI7oDE/wG24Hv0O4EAAA=>)</li></ul> | 3.90 ns | 5.64 ns | 1.45 | 0.07 | False | | |

![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_1.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_2.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_3.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_4.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_5.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_6.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_7.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_8.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_9.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_10.png>)
![graph](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress_11.png>)
[Test Report](<https://pvscmduploaddev.blob.core.windows.net/autofilereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/StoreBlock.LocalAddress.html>)

### Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

<details>
   <summary>Repro Steps</summary>

#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0`
- CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0`
- AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false`
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder

Linux
```cmd
# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'StoreBlock.LocalAddress*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter StoreBlock.LocalAddress* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'StoreBlock.LocalAddress*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter StoreBlock.LocalAddress* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### StoreBlock.LocalAddress.InitBlockAllOnes16 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllOnes128 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllOnes64 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllOnes32 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.CopyBlock128 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllZeros128 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllZeros64 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.CopyBlock32 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllZeros32 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.CopyBlock64 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.LocalAddress.InitBlockAllOnes8 #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.41 ns 3.61 ns 8.90 0.27 True
0.38 ns 3.71 ns 9.87 0.27 True
0.40 ns 10.15 ns 25.41 0.35 True
0.11 ns 1.84 ns 16.17 0.63 True
3.70 ns 15.09 ns 4.08 0.24 True
2.88 ns 12.75 ns 4.43 0.10 True
8.15 ns 19.50 ns 2.39 0.01 True
22.03 ns 133.04 ns 6.04 0.09 True
0.32 ns 9.86 ns 30.72 0.56 True
1.01 ns 13.05 ns 12.96 0.20 True
2.90 ns 73.36 ns 25.30 0.10 True
2.93 ns 12.68 ns 4.32 0.12 True
3.62 ns 34.47 ns 9.53 0.21 True

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Plane* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Plane* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Numerics.Tests.Perf_Plane.EqualityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.InequalityOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.EqualsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.NormalizeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.CreateFromScalarXYZDBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.CreateFromVector4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.TransformByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.DotNormalBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.DotBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.CreateFromVector3WithScalarDBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Plane.TransformByQuaternionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
42.04 ns 127.84 ns 3.04 0.10 True
81.42 ns 256.41 ns 3.15 0.15 True
32.16 ns 92.34 ns 2.87 0.07 True
924.18 ns 1.26 μs 1.36 0.01 False
7.02 ns 11.28 ns 1.61 0.12 False
91.79 ns 188.38 ns 2.05 0.03 False
5.82 ns 36.87 ns 6.33 0.18 True
350.21 ns 1.51 μs 4.30 0.05 True
722.29 ns 842.18 ns 1.17 0.01 True
177.67 ns 769.63 ns 4.33 0.09 True
71.13 ns 122.23 ns 1.72 0.04 True
594.94 ns 732.04 ns 1.23 0.01 False
72.47 ns 138.62 ns 1.91 0.01 True
44.20 ns 67.19 ns 1.52 0.02 True
145.57 ns 379.85 ns 2.61 0.03 True
6.25 ns 22.77 ns 3.64 0.15 True
33.75 ns 38.86 ns 1.15 0.06 False
31.30 ns 33.85 ns 1.08 0.02 False
9.60 μs 11.16 μs 1.16 0.06 False
15.02 ns 21.57 ns 1.44 0.08 True
18.60 ns 23.53 ns 1.27 0.06 False
1.49 μs 2.77 μs 1.86 0.04 True
31.86 ns 34.18 ns 1.07 0.06 False
12.76 ns 26.63 ns 2.09 0.13 True
36.21 ns 39.68 ns 1.10 0.08 False
9.70 ns 16.52 ns 1.70 0.12 True
19.05 ns 25.93 ns 1.36 0.06 False
72.44 ns 130.59 ns 1.80 0.06 True
42.49 ns 67.64 ns 1.59 0.03 True
3.82 μs 5.99 μs 1.57 0.00 True
8.16 ns 11.18 ns 1.37 0.17 False
452.71 ns 2.21 μs 4.88 0.04 True
54.77 ns 177.46 ns 3.24 0.18 True
30.31 ns 54.73 ns 1.81 0.10 True
36.64 ns 91.03 ns 2.48 0.09 True
329.99 ns 582.75 ns 1.77 0.01 False
788.46 ns 1.04 μs 1.32 0.01 True
29.74 ns 54.34 ns 1.83 0.08 True
29.73 ns 34.89 ns 1.17 0.08 False
123.14 ns 146.72 ns 1.19 0.03 False
176.78 ns 325.84 ns 1.84 0.06 True
40.11 ns 67.51 ns 1.68 0.15 False
6.07 ns 22.98 ns 3.79 0.17 True
40.62 ns 69.28 ns 1.71 0.13 True
15.67 ns 22.15 ns 1.41 0.08 True
38.65 ns 108.33 ns 2.80 0.07 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph ![graph](<https://pvscmduploaddev.blob.core.windows.net/autofil

performanceautofiler[bot] commented 7 months ago

ereport/autofilereports/02_27_2024/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Tests.Perf_String_26.png>) graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_String* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_String* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_String.Trim(s: " Te st ") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str_str(size: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.GetChars(size: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N} {0:X} !!", o: 8) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.TrimStart_CharArr(s: "Test", c: [' ', ' ']) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Trim(s: "Test") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str(size: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Join_Enumerable #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.GetChars(size: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Join_Array #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str_str_str(size: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.TrimStart(s: "Test") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.PadLeft(n: 18) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_CharEnumerable #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' ']) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' ']) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', ' ']) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.CtorCharCount(size: 10) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Join_List #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str_str_str(size: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' ']) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str_str(size: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str(size: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.TrimEnd(s: "Test ") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Trim(s: " Test") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Format_MultipleArgs #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: RemoveEmptyEntries) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.TrimStart(s: " Test") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.IndexerCheckPathLength #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Interpolation_MultipleArgs(c: '1', s: "Foo") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.IndexerCheckBoundCheckHoist #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.TrimEnd(s: "Test") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.IndexerCheckLengthHoisting #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_String.Trim(s: "Test ") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.86 μs 15.36 μs 3.16 0.01 True
21.76 μs 42.54 μs 1.96 0.09 True
7.83 μs 25.14 μs 3.21 0.00 True
18.62 μs 44.96 μs 2.41 0.07 True
18.68 μs 34.92 μs 1.87 0.04 True
31.63 μs 116.03 μs 3.67 0.03 True
4.51 μs 18.24 μs 4.05 0.00 True
5.41 μs 31.71 μs 5.86 0.02 True
41.11 μs 70.36 μs 1.71 0.07 True
16.47 μs 105.54 μs 6.41 0.09 True
5.39 μs 26.91 μs 4.99 0.02 True
15.32 μs 22.91 μs 1.50 0.11 False
5.90 μs 17.56 μs 2.98 0.01 True
21.12 μs 69.82 μs 3.31 0.06 True
30.41 μs 107.28 μs 3.53 0.01 True
36.04 μs 58.92 μs 1.63 0.04 True
22.13 μs 51.99 μs 2.35 0.11 True
42.58 μs 67.20 μs 1.58 0.03 True
4.42 μs 44.04 μs 9.96 0.01 True
29.49 μs 107.02 μs 3.63 0.02 True
4.83 μs 24.24 μs 5.02 0.02 True
28.96 μs 110.45 μs 3.81 0.02 True
22.17 μs 67.90 μs 3.06 0.02 True
62.32 μs 74.90 μs 1.20 0.04 True
49.77 μs 63.62 μs 1.28 0.12 False
59.94 μs 72.65 μs 1.21 0.10 False
16.97 μs 34.97 μs 2.06 0.01 True
34.16 μs 54.09 μs 1.58 0.06 True
14.24 μs 20.72 μs 1.46 0.09 False
41.88 μs 63.51 μs 1.52 0.03 True
39.39 μs 41.64 μs 1.06 0.02 False
18.71 μs 45.34 μs 2.42 0.06 True
5.35 μs 32.82 μs 6.14 0.12 True
54.56 μs 206.30 μs 3.78 0.03 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.MathBenchmarks.Single*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.MathBenchmarks.Single* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.MathBenchmarks.Single*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.MathBenchmarks.Single* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.MathBenchmarks.Single.Max #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Exp #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.CopySign #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Log2 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Cos #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.TanPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Abs #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Floor #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Exp10 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Hypot #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Ceiling #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.ScaleB #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Min #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Exp2 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.SinCos #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Log10P1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.ExpM1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Exp10M1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Round #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.SinPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Sqrt #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.CosPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Exp2M1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.AcosPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.AsinPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.AtanPi #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Sin #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Log10 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.ILogB #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Pow #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Log #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.Log2P1 #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.FusedMultiplyAdd #### ETL Files #### Histogram #### JIT Disasms ### System.MathBenchmarks.Single.SinCosPi #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.07 ms 48.17 ms 3.99 0.02 True
3.92 ms 66.19 ms 16.89 0.02 True
7.91 ms 33.98 ms 4.29 0.03 True
7.73 ms 33.75 ms 4.37 0.02 True
57.07 ms 99.34 ms 1.74 0.00 True
8.52 ms 67.38 ms 7.91 0.02 True
13.07 ms 49.63 ms 3.80 0.00 True
6.93 ms 32.29 ms 4.66 0.03 True
6.65 ms 32.34 ms 4.86 0.05 True
4.54 ms 67.43 ms 14.84 0.02 True
60.36 ms 169.23 ms 2.80 0.00 True
9.25 ms 68.70 ms 7.43 0.01 True
12.86 ms 49.99 ms 3.89 0.01 True
60.53 ms 164.39 ms 2.72 0.00 True
4.83 ms 67.62 ms 13.99 0.02 True
57.30 ms 101.06 ms 1.76 0.01 True
60.70 ms 169.38 ms 2.79 0.00 True
9.07 ms 68.58 ms 7.56 0.01 True
4.05 ms 66.33 ms 16.39 0.02 True
12.38 ms 49.05 ms 3.96 0.01 True
56.75 ms 99.89 ms 1.76 0.01 True
61.07 ms 166.97 ms 2.73 0.00 True
8.66 ms 67.38 ms 7.78 0.01 True
57.18 ms 101.73 ms 1.78 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Perf_Strings* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Perf_Strings* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: NoneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: OneEscaped) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: AllEscaped) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
68.12 ns 78.91 ns 1.16 0.05 False
254.22 ns 316.74 ns 1.25 0.05 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Concurrent.Count<Int32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Concurrent.Count<Int32>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Concurrent.Count<Int32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Concurrent.Count<Int32>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.Concurrent.Count<Int32>.Bag(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.40 ns 19.35 ns 48.16 0.68 True
3.46 ns 28.30 ns 8.18 0.08 True
3.32 ns 18.67 ns 5.62 0.24 True
3.23 ns 10.25 ns 3.18 0.34 True
3.31 ns 10.18 ns 3.07 0.36 True
3.34 ns 21.56 ns 6.46 0.27 True
0.38 ns 6.50 ns 17.24 0.51 True
5.14 ns 120.76 ns 23.51 0.26 True
0.30 ns 12.95 ns 42.98 0.53 True
3.98 ns 29.58 ns 7.44 0.16 True
3.15 ns 10.22 ns 3.24 0.20 True
2.92 ns 10.31 ns 3.53 0.09 True
3.18 ns 14.24 ns 4.48 0.33 True
2.95 ns 11.22 ns 3.80 0.12 True
2.87 ns 11.90 ns 4.15 0.14 True
3.72 ns 29.56 ns 7.95 0.15 True
3.29 ns 10.13 ns 3.08 0.27 True
3.65 ns 27.37 ns 7.50 0.42 True
3.16 ns 28.40 ns 8.98 0.28 True
3.20 ns 10.12 ns 3.16 0.21 True
3.20 ns 10.04 ns 3.13 0.28 True
3.49 ns 14.05 ns 4.02 0.25 True
3.28 ns 10.41 ns 3.17 0.37 True
3.04 ns 10.33 ns 3.40 0.32 True
5.02 ns 120.80 ns 24.07 0.28 True
3.28 ns 10.33 ns 3.15 0.32 True
3.34 ns 18.90 ns 5.66 0.28 True
2.86 ns 11.24 ns 3.93 0.13 True
0.25 ns 13.31 ns 53.54 0.55 True
3.14 ns 33.21 ns 10.58 0.22 True
3.12 ns 14.27 ns 4.58 0.28 True
3.00 ns 11.14 ns 3.71 0.12 True
2.87 ns 11.10 ns 3.87 0.10 True
3.48 ns 19.91 ns 5.73 0.17 True
3.46 ns 41.34 ns 11.95 0.33 True
3.99 ns 10.18 ns 2.55 0.35 True
4.84 ns 97.82 ns 20.22 0.37 True
3.23 ns 21.88 ns 6.78 0.18 True
3.07 ns 13.86 ns 4.52 0.24 True
0.35 ns 2.15 ns 6.21 0.66 True
2.93 ns 9.77 ns 3.33 0.08 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParal
performanceautofiler[bot] commented 7 months ago

lel=false /m:1

Build

dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

Run

dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector4* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200


Windows
```cmd
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector4* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Numerics.Tests.Perf_Vector4.DistanceBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DivideOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.AddOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.LengthBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DistanceSquaredBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.SubtractOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.CreateFromScalar #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MinBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.UnitWBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.OneBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.NegateOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.ClampBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.NegateBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.AddFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.CreateFromVector3WithScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MultiplyFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DivideBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.UnitYBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.SquareRootBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.MaxBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.UnitZBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.UnitXBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.AbsBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.LerpBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.SubtractFunctionBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.DivideByScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.CreateFromVector2WithScalarBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.LengthSquaredBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Numerics.Tests.Perf_Vector4.CreateFromScalarXYZWBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
41.47 ns 44.88 ns 1.08 0.04 False
35.26 ns 51.85 ns 1.47 0.00 True
60.83 ns 73.42 ns 1.21 0.01 True
36.16 ns 52.02 ns 1.44 0.00 True
12.42 ns 288.70 ns 23.25 0.12 True
2.64 ns 8.42 ns 3.19 0.11 True
10.31 ns 12.26 ns 1.19 0.12 False
14.12 ns 22.22 ns 1.57 0.02 True
103.19 ns 127.62 ns 1.24 0.02 True
3.67 ns 10.05 ns 2.74 0.19 True
41.85 ns 45.01 ns 1.08 0.02 False
20.52 ns 397.04 ns 19.35 0.07 True
6.73 ns 18.19 ns 2.70 0.07 True
104.24 ns 124.33 ns 1.19 0.04 False
13.75 ns 22.32 ns 1.62 0.02 True
7.58 ns 38.59 ns 5.09 0.04 True
61.53 ns 71.79 ns 1.17 0.02 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Perf_Ascii*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Perf_Ascii* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Perf_Ascii*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Perf_Ascii* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Perf_Ascii.ToLower_Bytes(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.FromUtf16(Size: 128) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Bytes(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Bytes(Size: 128) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToLowerInPlace_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Chars(Size: 128) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUtf16(Size: 128) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.FromUtf16(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Chars(Size: 128) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToUtf16(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
83.38 ns 104.48 ns 1.25 0.02 True
16.62 ns 67.04 ns 4.03 0.24 True
24.53 ns 76.03 ns 3.10 0.08 True
29.70 ns 78.20 ns 2.63 0.17 True
4.17 ns 10.16 ns 2.43 0.03 True
12.62 ns 18.90 ns 1.50 0.15 True
13.28 ns 19.03 ns 1.43 0.16 True
10.54 ns 17.01 ns 1.61 0.20 True
7.89 ns 57.22 ns 7.25 0.09 True
183.08 ns 268.87 ns 1.47 0.02 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Net.Primitives.Tests.IPAddressPerformanceTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Net.Primitives.Tests.IPAddressPerformanceTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Net.Primitives.Tests.IPAddressPerformanceTests.CtorAndToString(address: [143, 24, 20, 36]) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...]) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...]) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 143.24.20.36) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 143.24.20.36) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [143, 24, 20, 36]) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [143, 24, 20, 36]) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516) #### ETL Files #### Histogram #### JIT Disasms ### System.Net.Primitives.Tests.IPAddressPerformanceTests.CtorAndToString(address: [16, 32, 48, 64, 80, ...]) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.36 ns 19.89 ns 1.29 0.05 True
69.26 ns 95.12 ns 1.37 0.02 True
51.58 ns 80.89 ns 1.57 0.00 True
12.64 ns 16.98 ns 1.34 0.01 True
15.31 ns 17.83 ns 1.16 0.09 False
12.68 ns 15.60 ns 1.23 0.04 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.ReadOnlySequence*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.ReadOnlySequence* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.ReadOnlySequence*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.ReadOnlySequence* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Single) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Single) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple) #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Single) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
686.94 ns 772.05 ns 1.12 0.08 False
486.44 ns 650.17 ns 1.34 0.01 False
680.22 ns 767.90 ns 1.13 0.08 False
656.25 ns 752.99 ns 1.15 0.06 True
707.43 ns 817.33 ns 1.16 0.04 True
72.82 ns 89.88 ns 1.23 0.07 True
431.38 ns 560.97 ns 1.30 0.04 False
744.93 ns 815.63 ns 1.09 0.02 True
378.73 ns 773.38 ns 2.04 0.02 True
685.00 ns 768.63 ns 1.12 0.05 False
759.25 ns 802.32 ns 1.06 0.05 False
739.69 ns 833.59 ns 1.13 0.06 False
731.30 ns 794.00 ns 1.09 0.02 False
601.67 ns 664.60 ns 1.10 0.06 False
773.79 ns 967.63 ns 1.25 0.05 True
569.75 ns 718.16 ns 1.26 0.02 False
136.91 ns 177.89 ns 1.30 0.05 True
48.03 ns 58.41 ns 1.22 0.10 False
679.27 ns 740.32 ns 1.09 0.02 False
136.64 ns 191.11 ns 1.40 0.08 True
73.20 ns 90.79 ns 1.24 0.07 True
47.85 ns 52.92 ns 1.11 0.09 False
71.62 ns 76.19 ns 1.06 0.05 False
390.17 ns 763.19 ns 1.96 0.01 True
644.11 ns 681.20 ns 1.06 0.05 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Enum*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Enum* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Enum*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Enum* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.StringFormat(value: 32) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoString(value: Red) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoString(value: 32) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.GetValuesAsUnderlyingType_Generic #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.StringFormat(value: Red) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: Saturday) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: 42) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: Red, Green) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red, Green) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: Saturday) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: Red) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoString(value: Red, Green) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.StringFormat(value: Red, Green) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.Parse_Flags(text: "Red") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_Format_Flags_Large(value: All, format: "d") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.GetValuesAsUnderlyingType_NonGeneric #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.ToString_Flags(value: 36) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
53.80 ns 76.73 ns 1.43 0.03 True
65.81 ns 91.16 ns 1.39 0.06 False
6.48 ns 11.25 ns 1.74 0.15 True
9.70 ns 39.32 ns 4.05 0.15 True
35.23 ns 43.00 ns 1.22 0.03 False
7.14 μs 8.35 μs 1.17 0.07 False
193.55 ns 8.10 μs 41.84 0.02 True
330.99 ns 1.46 μs 4.41 0.01 True
190.55 ns 1.14 μs 6.01 0.00 True
134.10 ns 405.10 ns 3.02 0.02 True
58.12 ns 65.78 ns 1.13 0.04 False
9.06 ns 11.44 ns 1.26 0.10 False
6.38 ns 11.56 ns 1.81 0.16 True
2.62 ns 4.87 ns 1.86 0.12 True
32.28 ns 53.81 ns 1.67 0.08 True
32.51 ns 34.94 ns 1.07 0.04 False
52.42 ns 91.23 ns 1.74 0.03 False
5.83 ns 10.64 ns 1.83 0.22 False
164.85 ns 1.09 μs 6.64 0.02 True
215.21 ns 431.73 ns 2.01 0.02 True
29.63 ns 53.96 ns 1.82 0.06 True
32.24 ns 53.83 ns 1.67 0.08 True
24.15 ns 47.72 ns 1.98 0.17 True
166.48 ns 1.09 μs 6.57 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Tests.Perf_BitArray* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.Tests.Perf_BitArray* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayGet(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayGet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayCopyToBoolArray(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySet(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayCopyToBoolArray(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.20 μs 8.35 μs 1.99 0.02 True
38.80 μs 81.67 μs 2.10 0.02 True
44.94 μs 94.69 μs 2.11 0.01 True
7.59 μs 13.90 μs 1.83 0.02 True
528.94 ns 999.02 ns 1.89 0.06 True
3.79 ms 7.92 ms 2.09 0.01 True
9.45 μs 18.24 μs 1.93 0.02 True
8.68 μs 15.27 μs 1.76 0.02 True
29.28 μs 69.89 μs 2.39 0.01 True
4.56 μs 9.05 μs 1.98 0.01 True
37.93 μs 84.85 μs 2.24 0.00 True
8.45 μs 16.30 μs 1.93 0.04 True
35.05 μs 75.30 μs 2.15 0.01 True
3.43 ms 7.26 ms 2.12 0.01 True
20.53 μs 59.89 μs 2.92 0.01 True
620.60 ns 1.05 μs 1.68 0.03 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_ParseThenWrite* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_ParseThenWrite* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.62 ns 20.42 ns 5.63 0.08 True
3.05 ns 50.23 ns 16.48 0.05 True
2.86 ns 11.42 ns 4.00 0.08 True
3.09 ns 30.25 ns 9.80 0.07 True
3.61 ns 11.32 ns 3.14 0.09 True
3.08 ns 11.90 ns 3.86 0.14 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertDoubleToULongBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertFloatToUIntBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertLongToDoubleBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertUIntToFloatBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertDoubleToLongBenchmark #### ETL Files #### Histogram #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertULongToDoubleBenchmark #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
359.12 ns 2.17 μs 6.05 0.03 True
6.53 μs 10.77 μs 1.65 0.11 True
2.38 μs 2.80 μs 1.18 0.14 False
1.82 μs 2.90 μs 1.59 0.12 False
4.91 μs 6.26 μs 1.27 0.10 False
0.24 ns 679.54 ns 2856.22 0.43 True
18.21 μs 19.19 μs 1.05 0.09 False
11.09 μs 12.20 μs 1.10 0.05 False
38.89 μs 48.50 μs 1.25 0.09 True
3.82 μs 4.51 μs 1.18 0.07 False
1.34 ns 790.10 ns 591.06 0.27 True
2.90 μs 3.94 μs 1.36 0.08 False
44.72 μs 49.06 μs 1.10 0.07 False
3.76 μs 4.19 μs 1.11 0.04 False
2.25 ns 457.87 ns 203.62 0.10 True
2.03 μs 2.56 μs 1.27 0.07 False
19.86 μs 21.84 μs 1.10 0.03 False
2.63 μs 3.41 μs 1.30 0.07 True
5.04 μs 6.01 μs 1.19 0.04 False
3.56 μs 4.63 μs 1.30 0.08 False
2.84 μs 3.09 μs 1.09 0.12 False
19.47 μs 22.60 μs 1.16 0.08 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IterateForEach<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.IterateForEach<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IterateForEach<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.IterateForEach<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.IterateForEach<String>.ImmutableArray(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.List(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.Array(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.Queue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ReadOnlySpan(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.Dictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.Stack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.Span(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.LinkedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ConcurrentBag(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.FrozenDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.IEnumerable(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.IterateForEach<String>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
134.32 ns 270.55 ns 2.01 0.03 True
205.61 ns 350.55 ns 1.70 0.02 True
164.15 ns 174.86 ns 1.07 0.01 True
161.21 ns 170.83 ns 1.06 0.01 True
218.54 ns 485.61 ns 2.22 0.04 True
213.33 ns 322.17 ns 1.51 0.00 True
207.51 ns 354.58 ns 1.71 0.01 True
1.49 μs 2.75 μs 1.85 0.05 True
216.08 ns 358.99 ns 1.66 0.01 True
135.81 ns 272.32 ns 2.01 0.03 True
194.46 ns 213.76 ns 1.10 0.01 True
222.42 ns 365.34 ns 1.64 0.01 True
138.30 ns 273.30 ns 1.98 0.02 True
189.75 ns 214.49 ns 1.13 0.02 True
199.13 ns 292.95 ns 1.47 0.01 True
195.71 ns 217.30 ns 1.11 0.03 True
224.44 ns 366.76 ns 1.63 0.02 True
247.74 ns 358.29 ns 1.45 0.02 True
139.27 ns 271.81 ns 1.95 0.05 True
215.79 ns 361.46 ns 1.68 0.02 True
206.99 ns 356.64 ns 1.72 0.02 True
264.75 ns 366.23 ns 1.38 0.02 True
477.38 ns 705.18 ns 1.48 0.02 True
216.22 ns 307.20 ns 1.42 0.01 True
186.61 ns 209.60 ns 1.12 0.00 True
956.53 μs 2.00 ms 2.09 0.00 True
950.91 μs 2.07 ms 2.18 0.00 True
1.48 μs 2.72 μs 1.83 0.06 True
207.35 ns 349.42 ns 1.69 0.02 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Single*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Single* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Single*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Single* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.Parse(value: "12345") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.TryParse(value: "12345") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "F50") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToString(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.Parse(value: "3.4028235E+38") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToString(value: -3.4028235E+38) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "F50") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.IsNaN(value: NaN) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.IsNaN(value: 0) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "F50") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Single.ToString(value: 3.4028235E+38) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.70 ns 7.22 ns 1.95 0.04 True
4.74 ns 14.80 ns 3.12 0.02 True
4.40 ns 14.66 ns 3.33 0.03 True
4.25 ns 14.92 ns 3.51 0.01 True
2.87 ns 6.47 ns 2.26 0.07 True
3.37 ns 6.25 ns 1.86 0.02 True
4.58 ns 14.89 ns 3.25 0.05 True
3.75 ns 7.11 ns 1.90 0.05 True
4.73 ns 14.74 ns 3.12 0.04 True
4.36 ns 14.64 ns 3.36 0.02 True
4.47 ns 14.91 ns 3.34 0.03 True
4.39 ns 14.91 ns 3.40 0.02 True
3.29 ns 6.52 ns 1.98 0.09 True
3.72 ns 7.20 ns 1.94 0.07 True
3.64 ns 6.26 ns 1.72 0.03 False
3.67 ns 7.15 ns 1.95 0.06 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Slice<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Slice<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Slice<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Slice<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Memory.Slice<String>.MemoryStart #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.MemoryStartSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.MemorySpanStart #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlyMemorySpanStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlySpanStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlySpanStart #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.MemoryStartLengthSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlyMemoryStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlyMemoryStartSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlyMemorySpanStart #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlyMemoryStartLengthSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.MemorySpanStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.SpanStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.ReadOnlyMemoryStart #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.SpanStart #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Slice<String>.MemoryStartLength #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.69 μs 18.38 μs 1.25 0.04 True
1.02 ms 1.10 ms 1.07 0.07 False
997.24 μs 1.10 ms 1.10 0.05 False
6.89 μs 8.28 μs 1.20 0.04 False
43.39 μs 55.05 μs 1.27 0.02 True
54.25 μs 69.26 μs 1.28 0.02 True
941.03 μs 1.07 ms 1.14 0.08 False
6.97 μs 9.59 μs 1.38 0.08 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.CreateAddAndRemove<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.CreateAddAndRemove<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.CreateAddAndRemove<String>.LinkedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.Stack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.Dictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.Queue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
405.34 ns 456.92 ns 1.13 0.07 False
312.69 ns 336.49 ns 1.08 0.09 False
252.43 ns 272.06 ns 1.08 0.21 True
22.80 ns 25.65 ns 1.13 0.03 False
56.12 ns 60.51 ns 1.08 0.03 False
233.18 ns 253.45 ns 1.09 0.07 True
365.37 ns 389.08 ns 1.06 0.03 True
1.84 μs 1.97 μs 1.07 0.25 False
138.77 ns 176.64 ns 1.27 0.07 False
111.07 ns 125.74 ns 1.13 0.19 False
148.84 ns 165.02 ns 1.11 0.11 False
331.54 ns 357.47 ns 1.08 0.11 False
97.74 ns 108.32 ns 1.11 0.09 False
76.61 ns 109.36 ns 1.43 0.15 True
354.83 ns 384.71 ns 1.08 0.10 False
239.09 ns 276.48 ns 1.16 0.07 True
221.15 ns 241.09 ns 1.09 0.10 True
556.62 ns 605.78 ns 1.09 0.06 False
349.11 ns 373.79 ns 1.07 0.07 True
366.72 ns 425.82 ns 1.16 0.19 False
195.10 ns 218.32 ns 1.12 0.10 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Reflection.Invoke*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Reflection.Invoke* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Reflection.Invoke*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Reflection.Invoke* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_Get_class #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_GetStatic_struct #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Ctor0_NoParams #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.StaticMethod4_int_string_struct_class #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_SetStatic_struct #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_Get_struct #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Ctor4_ActivatorCreateInstance #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Property_Set_class #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Property_Set_int #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_Set_int #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Method0_NoParms #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_Get_int #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_GetStatic_int #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_SetStatic_int #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.StaticMethod5_ByRefParams_int_string_struct_class_bool #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_Set_struct #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_Set_class #### ETL Files #### Histogram #### JIT Disasms ### System.Reflection.Invoke.Field_GetStatic_class #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
647.30 μs 683.61 μs 1.06 0.07 False
610.29 μs 683.83 μs 1.12 0.06 False
12.88 μs 19.14 μs 1.49 0.01 True
573.96 μs 643.59 μs 1.12 0.06 False
11.37 μs 17.36 μs 1.53 0.11 False
40.00 μs 49.50 μs 1.24 0.01 True
13.32 μs 18.65 μs 1.40 0.11 False
9.92 μs 15.79 μs 1.59 0.08 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.ContainsKeyFalse<String, String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.ContainsKeyFalse<String, String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.ContainsKeyFalse<String, String>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.FrozenDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.SortedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.Dictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.IDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Hashing

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
29.92 ns 58.57 ns 1.96 0.01 True
4.87 ns 12.35 ns 2.54 0.13 True
307.47 ns 527.33 ns 1.72 0.00 True
2.97 μs 5.20 μs 1.75 0.00 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Hashing*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Hashing* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Hashing*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Hashing* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Hashing.GetStringHashCode(BytesCount: 100) #### ETL Files #### Histogram #### JIT Disasms ### System.Hashing.GetStringHashCode(BytesCount: 10) #### ETL Files #### Histogram #### JIT Disasms ### System.Hashing.GetStringHashCode(BytesCount: 1000) #### ETL Files #### Histogram #### JIT Disasms ### System.Hashing.GetStringHashCode(BytesCount: 10000) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.08 ns 9.09 ns 4.37 0.13 True
56.44 ns 99.87 ns 1.77 0.05 True
2.35 ns 9.36 ns 3.99 0.16 True
59.06 ns 99.35 ns 1.68 0.05 True
313.81 ns 387.07 ns 1.23 0.02 True
1.93 ns 4.24 ns 2.20 0.17 True
79.40 ns 98.39 ns 1.24 0.01 True
103.70 ns 144.08 ns 1.39 0.01 True
313.29 ns 382.77 ns 1.22 0.03 True
2.72 ns 4.14 ns 1.52 0.04 True
2.30 ns 8.56 ns 3.72 0.11 True
88.95 ns 110.18 ns 1.24 0.01 True
2.56 ns 9.61 ns 3.75 0.16 True

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_DateTime*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_DateTime* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_DateTime*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_DateTime* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_DateTime.Year #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ParseR #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.Month #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ParseO #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ToString(format: "G") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ObjectEquals #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ToString(format: "s") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ToString(format: "o") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ToString(format: null) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.op_Subtraction #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.DayOfYear #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.ToString(format: "r") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_DateTime.Day #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.07 ns 14.21 ns 1.18 0.17 False
34.40 ns 54.25 ns 1.58 0.05 True
11.71 ns 41.93 ns 3.58 0.09 True
38.26 ns 80.03 ns 2.09 0.03 True
12.16 ns 15.40 ns 1.27 0.18 False
138.20 ns 1.70 μs 12.29 0.09 True
36.28 ns 592.42 ns 16.33 0.04 True
259.67 ns 1.97 μs 7.58 0.06 True
108.19 ns 1.09 μs 10.05 0.09 True
38.51 ns 81.26 ns 2.11 0.02 True
233.68 ns 1.79 μs 7.65 0.03 True
12.08 ns 15.41 ns 1.28 0.16 False
34.07 ns 95.78 ns 2.81 0.04 True
7.87 ns 10.17 ns 1.29 0.25 False
7.99 ns 10.15 ns 1.27 0.25 False
253.06 ns 2.02 μs 7.99 0.02 True
48.20 ns 109.28 ns 2.27 0.03 True
244.73 ns 1.79 μs 7.31 0.06 True
12.05 ns 15.55 ns 1.29 0.20 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Tests.Perf_Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Tests.Perf_Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Tests.Perf_Encoding.GetEncoder(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetEncoder(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetEncoder(size: 512, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetEncoder(size: 16, encName: "ascii") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.48 secs 1.85 secs 1.26 0.02 True
656.74 ms 856.02 ms 1.30 0.05 True
1.25 secs 2.66 secs 2.12 0.02 True
293.22 ms 411.82 ms 1.40 0.04 True
935.57 ms 1.59 secs 1.70 0.02 True
1.39 secs 3.94 secs 2.84 0.00 True
1.18 secs 1.71 secs 1.45 0.04 True
709.57 ms 1.01 secs 1.43 0.02 True
1.39 secs 1.83 secs 1.31 0.03 True
1.14 secs 1.69 secs 1.48 0.01 True
272.06 ms 741.66 ms 2.73 0.03 True
319.91 ms 886.69 ms 2.77 0.04 True

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'ByteMark*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter ByteMark* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'ByteMark*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter ByteMark* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### ByteMark.BenchAssignRectangular #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchNeural #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchLUDecomp #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchFourier #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchNumericSortRectangular #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchEmFloat #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchNumericSortJagged #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchNeuralJagged #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchStringSort #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchAssignJagged #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchEmFloatClass #### ETL Files #### Histogram #### JIT Disasms ### ByteMark.BenchIDEAEncryption #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.45 μs 7.67 μs 3.13 0.01 True
11.50 μs 28.74 μs 2.50 0.02 True
2.45 μs 7.69 μs 3.14 0.01 True
2.39 μs 7.46 μs 3.12 0.01 True
2.40 μs 7.44 μs 3.10 0.01 True
2.45 μs 7.69 μs 3.13 0.01 True
113.52 μs 665.29 μs 5.86 0.01 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_EnumerateArray* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_EnumerateArray* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfNumbers) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: Json400KB) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.87 ns 16.16 ns 3.32 0.30 True
5.48 ns 14.62 ns 2.66 0.27 True
4.10 ns 13.80 ns 3.36 0.31 True
159.79 ns 926.68 ns 5.80 0.09 True
5.22 ns 9.64 ns 1.85 0.27 True
2.43 ns 4.88 ns 2.01 0.26 True
2.29 μs 8.37 μs 3.65 0.05 True
351.16 ns 790.93 ns 2.25 0.08 True
5.12 ns 6.95 ns 1.36 0.24 False
3.70 ns 11.80 ns 3.19 0.32 True
2.29 μs 9.00 μs 3.94 0.05 True
161.87 ns 932.13 ns 5.76 0.09 True
32.23 ns 70.63 ns 2.19 0.10 True
3.41 ns 4.66 ns 1.37 0.27 False
29.89 ns 66.81 ns 2.24 0.11 True
2.99 ns 6.85 ns 2.29 0.20 True
6.64 ns 16.22 ns 2.44 0.37 True
2.43 ns 4.37 ns 1.80 0.29 False
3.84 ns 9.75 ns 2.54 0.25 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Random*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Random* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Random*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Random* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Random.NextSingle #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_int_int #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextDouble #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextBytes_span_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_long_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextSingle_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextBytes_span #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.ctor_seeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_int_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextBytes #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextBytes_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_long_long #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_long #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_int_int_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_int #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.NextDouble_unseeded #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Random.Next_long_long_unseeded #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
283.06 μs 609.50 μs 2.15 0.02 True
6.97 μs 7.86 μs 1.13 0.04 False
7.37 μs 7.83 μs 1.06 0.04 False
17.92 μs 25.44 μs 1.42 0.01 True
10.24 μs 11.80 μs 1.15 0.06 False
17.95 μs 25.05 μs 1.40 0.01 True
283.58 μs 605.95 μs 2.14 0.01 True
10.06 μs 11.52 μs 1.15 0.03 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.Perf_FileStream* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.Perf_FileStream* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.IO.Tests.Perf_FileStream.Read(fileSize: 1048576, userBufferSize: 512, options: Asynchronous) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.Read(fileSize: 1024, userBufferSize: 1024, options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.Read(fileSize: 1024, userBufferSize: 1024, options: Asynchronous) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.ReadByte(fileSize: 1024, options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: Asynchronous) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.Read(fileSize: 1048576, userBufferSize: 512, options: None) #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.Perf_FileStream.ReadByte(fileSize: 1024, options: Asynchronous) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.97 ns 20.04 ns 2.24 0.14 True
11.44 ns 19.85 ns 1.74 0.03 True
52.38 ns 209.80 ns 4.01 0.10 True
8.97 ns 20.48 ns 2.28 0.12 True
34.94 ns 114.87 ns 3.29 0.10 True
9.66 ns 29.96 ns 3.10 0.16 True
9.13 ns 12.04 ns 1.32 0.03 True
15.76 ns 24.66 ns 1.56 0.08 True
9.13 ns 25.04 ns 2.74 0.16 True
7.56 ns 9.04 ns 1.20 0.02 True
12.35 ns 23.46 ns 1.90 0.04 False
10.02 ns 31.74 ns 3.17 0.13 True
26.92 ns 97.00 ns 3.60 0.10 True

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Microsoft.Extensions.Primitives.StringSegmentBenchmark* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Microsoft.Extensions.Primitives.StringSegmentBenchmark* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.StartsWith #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Trim #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.EndsWith #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.SubSegment #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Ctor_String #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
55.44 μs 97.92 μs 1.77 0.03 True
96.49 μs 331.98 μs 3.44 0.00 True
131.73 μs 280.78 μs 2.13 0.03 True
163.35 μs 334.22 μs 2.05 0.02 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "simple.xml") #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "deep.xml") #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml") #### ETL Files #### Histogram #### JIT Disasms ### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "names.xml") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
90.18 ns 214.46 ns 2.38 0.04 True
441.22 ns 692.17 ns 1.57 0.04 True
361.01 ns 575.88 ns 1.60 0.04 True
398.97 ns 478.43 ns 1.20 0.02 False
376.57 ns 465.11 ns 1.24 0.07 False
0.00 ns 353.30 ns ∞ 0.00 True
732.51 ns 867.35 ns 1.18 0.04 True
599.58 ns 981.43 ns 1.64 0.02 True
716.82 ns 872.63 ns 1.22 0.14 False
89.25 ns 180.03 ns 2.02 0.03 True
225.28 ns 251.05 ns 1.11 0.03 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Uri*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Uri* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Uri*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Uri* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Uri.UnescapeDataString(input: "abc%20def%20ghi%20") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.ParseAbsoluteUri #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.DnsSafeHost #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "http://xn--hst-sna.with.xn--nicode-2ya") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.UriBuilderReplacePort #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path?key1=value1&key2=value2&key3=value3&key4=value4") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path%20with%20escapable%20values?key=va%20lue") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "http://höst.with.ünicode") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.UnescapeDataString(input: "%E4%BD%A0%E5%A5%BD") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "https://contoso.com") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
27.49 ns 289.78 ns 10.54 0.18 True
18.41 ns 38.79 ns 2.11 0.10 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter PerfLabTests.BlockCopyPerf* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter PerfLabTests.BlockCopyPerf* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 1000) #### ETL Files #### Histogram #### JIT Disasms ### PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 100) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.77 ns 17.22 ns 9.74 0.14 True
5.33 ns 38.63 ns 7.25 0.18 True
1.66 ns 17.28 ns 10.41 0.13 True
69.70 ns 462.57 ns 6.64 0.10 True
72.75 ns 469.26 ns 6.45 0.10 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Char*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Char* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Char*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Char* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Char.GetUnicodeCategory(c: 'a') #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Char.GetUnicodeCategory(c: 'א') #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Char.GetUnicodeCategory(c: '.') #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Char.Char_IsLower(input: "Good afternoon, Constable!") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.81 ns 15.78 ns 1.23 0.04 True
3.56 ns 8.25 ns 2.31 0.09 True
4.72 ns 8.99 ns 1.91 0.08 True
6.04 ns 12.14 ns 2.01 0.06 True
7.18 ns 9.33 ns 1.30 0.01 True
2.97 ns 7.89 ns 2.65 0.08 True
12.86 ns 15.59 ns 1.21 0.06 False
4.46 ns 9.55 ns 2.14 0.09 True
4.96 ns 9.66 ns 1.95 0.09 True
3.05 ns 7.88 ns 2.58 0.07 True
6.71 ns 10.96 ns 1.63 0.01 True
4.66 ns 9.15 ns 1.96 0.06 True
6.69 ns 10.86 ns 1.62 0.01 True
3.39 ns 8.29 ns 2.45 0.08 True
8.26 ns 9.82 ns 1.19 0.01 True
8.33 ns 9.93 ns 1.19 0.03 True
4.13 ns 9.67 ns 2.34 0.14 True
3.56 ns 8.48 ns 2.39 0.06 True
3.30 ns 9.05 ns 2.74 0.07 True
6.06 ns 10.06 ns 1.66 0.08 True
4.97 ns 8.94 ns 1.80 0.09 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Constructors<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Constructors<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Constructors<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Constructors<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlySpanFromArray #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.SpanFromArray #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ArrayAsMemoryStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlyMemoryFromArray #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.MemoryMarshalCreateReadOnlySpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.SpanFromArrayStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ArrayAsSpanStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.MemoryMarshalCreateSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlySpanFromMemory #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ArrayAsSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.SpanFromMemory #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArray #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ArrayAsMemory #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.MemoryFromArray #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlyMemoryFromArrayStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromSpan #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlySpanFromArrayStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.MemoryFromArrayStartLength #### ETL Files #### Histogram #### JIT Disasms ### System.Memory.Constructors<String>.SpanImplicitCastFromArray #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.79 μs 8.44 μs 3.02 0.01 True
43.84 μs 54.05 μs 1.23 0.02 True
2.72 μs 8.33 μs 3.06 0.01 True
5.90 μs 16.35 μs 2.77 0.01 True
5.18 μs 16.16 μs 3.12 0.02 True
2.13 ms 2.37 ms 1.11 0.03 False
5.18 μs 16.24 μs 3.14 0.02 True
2.73 μs 8.32 μs 3.05 0.01 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_EnumerateObject* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Document.Tests.Perf_EnumerateObject* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: NumericProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: ObjectProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: StringProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: ObjectProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: NumericProperties) #### ETL Files #### Histogram #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
385.06 ms 1.10 secs 2.85 0.03 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Benchstone.BenchI.BenchE*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Benchstone.BenchI.BenchE* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Benchstone.BenchI.BenchE*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Benchstone.BenchI.BenchE* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### Benchstone.BenchI.BenchE.Test #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.67 μs 14.58 μs 1.07 0.02 False
593.88 ns 761.54 ns 1.28 0.07 False
480.80 ns 568.32 ns 1.18 0.15 False
749.63 ns 865.55 ns 1.15 0.07 False
1.03 μs 1.27 μs 1.23 0.09 False
1.23 μs 1.46 μs 1.19 0.09 False
1.28 μs 1.44 μs 1.13 0.02 False
1.26 μs 1.35 μs 1.08 0.09 False
501.85 ns 586.76 ns 1.17 0.09 False
745.22 ns 798.80 ns 1.07 0.03 False
328.16 ns 362.04 ns 1.10 0.03 False
482.60 ns 587.00 ns 1.22 0.08 False
641.13 ns 712.61 ns 1.11 0.08 False
3.84 μs 4.23 μs 1.10 0.05 False
589.13 ns 681.30 ns 1.16 0.07 False
374.99 ns 395.98 ns 1.06 0.04 False
671.78 ns 803.05 ns 1.20 0.04 False
664.05 ns 731.74 ns 1.10 0.09 False
654.90 ns 737.50 ns 1.13 0.09 False
1.30 μs 1.59 μs 1.22 0.04 False
2.46 μs 3.34 μs 1.36 0.08 False
660.09 ns 710.57 ns 1.08 0.06 False
620.17 ns 695.02 ns 1.12 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Linq.Tests.Perf_Enumerable* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Linq.Tests.Perf_Enumerable* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: List) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: Array) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.CastToBaseClass(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.IO.Tests.BinaryReaderTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.62 ns 26.65 ns 4.74 0.13 True
5.69 ns 26.32 ns 4.62 0.22 True
39.89 ns 64.54 ns 1.62 0.04 True
632.54 ns 3.06 μs 4.84 0.04 True
5.71 ns 26.44 ns 4.63 0.17 True
5.57 ns 26.56 ns 4.77 0.19 True
8.40 ns 29.50 ns 3.51 0.13 True
5.66 ns 26.92 ns 4.76 0.21 True
126.22 ns 347.14 ns 2.75 0.01 True
63.76 ns 140.46 ns 2.20 0.04 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryReaderTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.BinaryReaderTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryReaderTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.BinaryReaderTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.IO.Tests.BinaryReaderTests.ReadUInt16 #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadUInt32 #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadAsciiChar #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadLargeString #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadUInt64 #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadDouble #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadHalf #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadSingle #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadNonAsciiChar #### ETL Files #### Histogram #### JIT Disasms ### System.IO.Tests.BinaryReaderTests.ReadSmallString #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.72 ns 47.43 ns 2.84 0.04 True
142.86 ns 168.48 ns 1.18 0.02 True
86.01 ns 99.03 ns 1.15 0.07 False
82.79 ns 89.66 ns 1.08 0.07 False
82.81 ns 97.42 ns 1.18 0.08 False
34.43 ns 148.65 ns 4.32 0.03 True
82.59 ns 97.63 ns 1.18 0.08 False
86.32 ns 92.05 ns 1.07 0.07 False
123.29 ns 140.36 ns 1.14 0.00 True
44.72 ns 165.36 ns 3.70 0.05 True
39.88 ns 43.79 ns 1.10 0.02 False
85.10 ns 90.93 ns 1.07 0.08 False
81.96 ns 90.11 ns 1.10 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Int64*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Int64* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Int64*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Int64* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Int64.ToString(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.Parse(value: "-9223372036854775808") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryParse(value: "9223372036854775807") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.ToString(value: 9223372036854775807) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.Parse(value: "9223372036854775807") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.ToString(value: -9223372036854775808) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryFormat(value: 12345) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808") #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808") #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
21.82 μs 29.20 μs 1.34 0.04 True
551.34 μs 635.10 μs 1.15 0.06 False
23.11 μs 30.06 μs 1.30 0.03 True
3.43 μs 4.96 μs 1.45 0.12 False
3.11 μs 3.99 μs 1.28 0.04 False
2.60 μs 4.50 μs 1.73 0.07 True
85.16 μs 90.07 μs 1.06 0.00 False
20.05 μs 25.35 μs 1.26 0.02 True
3.75 μs 4.51 μs 1.20 0.14 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.AddGivenSize<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.AddGivenSize<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.AddGivenSize<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.AddGivenSize<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.AddGivenSize<String>.Dictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.IDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.Queue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.Stack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.List(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.AddGivenSize<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
150.99 ns 163.46 ns 1.08 0.04 False
4.38 μs 6.61 μs 1.51 0.14 True
162.10 ns 231.06 ns 1.43 0.03 True
68.12 ns 76.59 ns 1.12 0.10 False
82.14 ns 106.23 ns 1.29 0.08 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Environment*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Environment* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Environment*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Environment* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Tests.Perf_Environment.GetFolderPath(folder: System, option: None) #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Environment.GetEnvironmentVariables #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Environment.ExpandEnvironmentVariables #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Environment.GetEnvironmentVariable #### ETL Files #### Histogram #### JIT Disasms ### System.Tests.Perf_Environment.SetEnvironmentVariable #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.93 μs 20.98 μs 1.62 0.02 True
12.60 μs 19.99 μs 1.59 0.10 False
784.38 μs 943.16 μs 1.20 0.11 False
413.19 μs 606.92 μs 1.47 0.08 True
796.07 μs 970.96 μs 1.22 0.11 False
459.97 μs 514.81 μs 1.12 0.07 False
486.51 μs 537.58 μs 1.10 0.06 False
786.89 μs 990.45 μs 1.26 0.12 False
42.78 μs 50.66 μs 1.18 0.04 True

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.ContainsTrue<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.ContainsTrue<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.ContainsTrue<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.ContainsTrue<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.ContainsTrue<String>.FrozenSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.Span(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.List(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.Queue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.ContainsTrue<String>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in StoreBlock.AnyLocation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.95 ns 10.13 ns 1.13 0.08 False
0.81 ns 1.98 ns 2.45 0.05 False
1.21 ns 3.62 ns 2.98 0.21 True
5.32 ns 6.43 ns 1.21 0.08 False
4.62 ns 5.71 ns 1.24 0.04 False
4.33 ns 5.93 ns 1.37 0.07 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'StoreBlock.AnyLocation*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter StoreBlock.AnyLocation* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'StoreBlock.AnyLocation*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter StoreBlock.AnyLocation* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### StoreBlock.AnyLocation.InitBlockAllOnes128 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.AnyLocation.CopyBlock8 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.AnyLocation.InitBlockAllZeros128 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.AnyLocation.InitBlockAllOnes32 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.AnyLocation.InitBlockAllOnes8 #### ETL Files #### Histogram #### JIT Disasms ### StoreBlock.AnyLocation.InitBlockAllOnes16 #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
38.59 ns 50.64 ns 1.31 0.10 False
151.95 ns 181.24 ns 1.19 0.08 False
173.17 ns 193.48 ns 1.12 0.08 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Xml.Linq.Perf_XName*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Xml.Linq.Perf_XName* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Xml.Linq.Perf_XName*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Xml.Linq.Perf_XName* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Xml.Linq.Perf_XName.NonEmptyNameSpaceToString #### ETL Files #### Histogram #### JIT Disasms ### System.Xml.Linq.Perf_XName.CreateElementWithNamespace #### ETL Files #### Histogram #### JIT Disasms ### System.Xml.Linq.Perf_XName.CreateElementWithNamespaceImplicitOperator #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
548.30 μs 615.39 μs 1.12 0.04 False
6.08 μs 7.01 μs 1.15 0.04 False
537.05 μs 597.42 μs 1.11 0.04 False
14.39 μs 17.37 μs 1.21 0.03 False
5.14 μs 5.69 μs 1.11 0.07 False
40.82 μs 49.74 μs 1.22 0.02 True
204.03 μs 221.99 μs 1.09 0.01 False
39.67 μs 49.60 μs 1.25 0.01 True
1.22 μs 1.54 μs 1.26 0.06 False
13.21 μs 14.53 μs 1.10 0.02 True
4.18 μs 5.08 μs 1.21 0.06 False
4.16 μs 5.95 μs 1.43 0.05 True
7.78 μs 8.95 μs 1.15 0.01 True
8.97 μs 11.22 μs 1.25 0.06 True
565.75 μs 612.65 μs 1.08 0.06 False
27.72 μs 34.68 μs 1.25 0.03 False
4.69 μs 5.85 μs 1.25 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.CreateAddAndClear<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.CreateAddAndClear<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads [Baseline]() [Compare](< https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-cac820bf-96ee-474a-bfac-e3b7e147c068c84fe665e804fdbbf/7fa1de43-9f83-465d-8ad1-e5d82266c1bc.zip?sv=2021-08-06&se=2024-03-21T09%3A53%3A42Z&sr=c&sp=rl&sig=Vkr%2B8eI1wdklkiyz2h%2BHG8bycmoi7TRiECuqeFV947c%3D>) ### System.Collections.CreateAddAndClear<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ImmutableStack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ImmutableQueue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.IDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.Span(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ConcurrentQueue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.Stack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.List(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.LinkedList(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.Queue(Size: 512) #### ETL Files #### Histogram #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 7 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5e1caf14fd94b4ed85357002303c8935fedacb8b
Compare 9dc6ea62a4d195ae4559f4609a56933c61889756
Diff Diff
Configs CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.28 ns 27.55 ns 6.44 0.12 True
0.37 ns 9.57 ns 26.05 0.32 True
3.93 ns 29.71 ns 7.57 0.13 True
3.31 ns 9.95 ns 3.00 0.06 True
3.85 ns 16.87 ns 4.38 0.13 True
3.42 ns 13.00 ns 3.80 0.07 True
4.10 ns 10.91 ns 2.66 0.22 True
3.80 ns 17.28 ns 4.55 0.13 True
4.02 ns 25.92 ns 6.45 0.20 True
3.41 ns 6.47 ns 1.89 0.13 True
0.41 ns 2.21 ns 5.37 0.12 True
4.03 ns 16.35 ns 4.06 0.20 True
4.15 ns 10.95 ns 2.64 0.22 True
4.01 ns 13.08 ns 3.26 0.21 True
3.75 ns 19.52 ns 5.21 0.19 True
0.88 ns 23.38 ns 26.47 0.15 True
4.04 ns 9.91 ns 2.45 0.26 True
3.29 ns 8.31 ns 2.52 0.07 True
0.42 ns 3.61 ns 8.53 0.34 True
4.27 ns 11.47 ns 2.68 0.26 True
3.73 ns 10.28 ns 2.75 0.12 True
3.94 ns 10.29 ns 2.61 0.21 True
3.99 ns 10.25 ns 2.57 0.14 True
4.02 ns 10.30 ns 2.56 0.23 True
3.52 ns 12.67 ns 3.60 0.06 True
5.05 ns 27.30 ns 5.41 0.20 True
3.48 ns 11.15 ns 3.21 0.07 True
0.48 ns 3.39 ns 6.99 0.38 True
0.52 ns 3.48 ns 6.67 0.12 True
0.43 ns 8.17 ns 18.83 0.36 True
3.41 ns 9.86 ns 2.89 0.10 True
3.99 ns 13.20 ns 3.31 0.20 True
0.68 ns 14.34 ns 21.10 0.23 True
3.97 ns 10.26 ns 2.58 0.18 True
3.99 ns 10.25 ns 2.57 0.21 True
4.09 ns 10.29 ns 2.51 0.20 True
4.06 ns 32.66 ns 8.05 0.23 True
4.25 ns 33.85 ns 7.96 0.27 True
5.49 ns 123.17 ns 22.42 0.38 True
5.44 ns 115.47 ns 21.22 0.30 True
3.80 ns 16.56 ns 4.36 0.22 True
3.33 ns 9.86 ns 2.97 0.11 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/