[Perf] Linux/arm64: 315 Improvements on 3/1/2023 1:48:25 AM

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage

MIT License

9 stars 4 forks source link

[Perf] Linux/arm64: 315 Improvements on 3/1/2023 1:48:25 AM #13761

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes%3a%20100).html>)	4.40 μs	3.74 μs	0.85	0.03	False
[ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes%3a%20100).html>)	4.39 μs	3.66 μs	0.83	0.03	False
[ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes%3a%201000).html>)	34.27 μs	26.24 μs	0.77	0.09	False
[ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes%3a%201000).html>)	34.72 μs	26.22 μs	0.76	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Reader.Tests.Perf_Base64*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Reader.Tests.Perf_Base64*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 48.481268082086444 (T) = (0 -3683.990299731998) / Math.Sqrt((16130.880527788524 / (109)) + (991.0585753978896 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.18889483125723758 = (4541.939124173373 - 3683.990299731998) / 4541.939124173373 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 61.057473690445555 (T) = (0 -3676.201479552788) / Math.Sqrt((15601.576457769852 / (108)) + (341.2551186761707 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.19072057599227313 = (4542.561407711866 - 3676.201479552788) / 4542.561407711866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 58.56129040399435 (T) = (0 -26490.772701873986) / Math.Sqrt((1886237.6501402762 / (109)) + (54884.154734159725 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.26445729690342296 = (36015.27496683729 - 26490.772701873986) / 36015.27496683729 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 57.3184602005205 (T) = (0 -26467.285076469412) / Math.Sqrt((1871425.2667720492 / (109)) + (61569.97479684638 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.26398991807363154 = (35960.49256172722 - 26467.285076469412) / 35960.49256172722 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options%3a%20IgnoreCase%2c%20NonBacktracking).html>)	74.33 secs	60.22 secs	0.81	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 14.316412298371835 (T) = (0 -59055200353.794304) / Math.Sqrt((5.314629341242015E+18 / (109)) + (5.61642531636431E+18 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.1939243182012251 = (73262600134.5325 - 59055200353.794304) / 73262600134.5325 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>)	869.17 μs	720.64 μs	0.83	0.02	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>)	596.17 μs	491.76 μs	0.82	0.05	False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromStream(Mode%3a%20SourceGen).html>)	644.19 μs	524.15 μs	0.81	0.03	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>)	601.73 μs	495.69 μs	0.82	0.02	False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>)	878.87 μs	746.47 μs	0.85	0.06	False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromStream(Mode%3a%20Reflection).html>)	624.56 μs	519.16 μs	0.83	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.134436780100984 (T) = (0 -724244.2791895604) / Math.Sqrt((834454167.8676476 / (89)) + (24850163.410798818 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.1762058599253595 = (879156.8717931639 - 724244.2791895604) / 879156.8717931639 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 18.86504327651914 (T) = (0 -499412.04991802707) / Math.Sqrt((988184626.167459 / (89)) + (87287241.08285023 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.16059223037345502 = (594957.621300333 - 499412.04991802707) / 594957.621300333 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.97007325614024 (T) = (0 -528371.4418325807) / Math.Sqrt((788322715.8230438 / (89)) + (9039945.237300728 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.16303467804798505 = (631294.305719005 - 528371.4418325807) / 631294.305719005 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 28.504253369605543 (T) = (0 -498533.3087631723) / Math.Sqrt((1004131078.71324 / (89)) + (8232639.787104474 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.1690167938282721 = (599931.8699349831 - 498533.3087631723) / 599931.8699349831 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 26.003148023610482 (T) = (0 -725966.5257905786) / Math.Sqrt((789226692.2877406 / (89)) + (144420960.90435976 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.17051600099270994 = (875202.5676919637 - 725966.5257905786) / 875202.5676919637 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.49616483014167 (T) = (0 -524734.3182622005) / Math.Sqrt((766654577.8612329 / (89)) + (9263443.249048218 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.16058318011136866 = (625117.7077102398 - 524734.3182622005) / 625117.7077102398 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20SourceGen).html>)	662.41 μs	611.43 μs	0.92	0.02	False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20SourceGen).html>)	790.44 μs	706.12 μs	0.89	0.03	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20SourceGen).html>)	694.59 μs	617.27 μs	0.89	0.02	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>)	694.34 μs	614.90 μs	0.89	0.03	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>)	639.77 μs	587.53 μs	0.92	0.02	False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20Reflection).html>)	662.32 μs	608.78 μs	0.92	0.02	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>)	642.77 μs	592.51 μs	0.92	0.02	False

Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 16.62231072872593 (T) = (0 -616410.6295054945) / Math.Sqrt((361441599.0179486 / (89)) + (17431829.02162954 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.06644564714486012 = (660283.6006498095 - 616410.6295054945) / 660283.6006498095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.31202619859616 (T) = (0 -717503.5190902847) / Math.Sqrt((405959856.0908404 / (89)) + (104918258.50292596 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07456472462919114 = (775314.6418616809 - 717503.5190902847) / 775314.6418616809 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.64384355302179 (T) = (0 -620815.6644307082) / Math.Sqrt((276891062.3436528 / (89)) + (9671360.047307137 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10536559726583483 = (693932.2504627396 - 620815.6644307082) / 693932.2504627396 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.508300155810904 (T) = (0 -623226.6108142552) / Math.Sqrt((315785094.20529085 / (89)) + (76441072.06083438 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10183400553000437 = (693888.0058379619 - 623226.6108142552) / 693888.0058379619 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 10.374963180281686 (T) = (0 -597710.6464875344) / Math.Sqrt((456890997.5971396 / (89)) + (49805112.27945329 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.059816148692205855 = (635738.0481021025 - 597710.6464875344) / 635738.0481021025 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 15.601725722052185 (T) = (0 -615257.509018679) / Math.Sqrt((331262817.7999987 / (89)) + (26895359.239686467 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.06771625111347146 = (659946.6200644501 - 615257.509018679) / 659946.6200644501 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 8.288434070285088 (T) = (0 -598886.3951942156) / Math.Sqrt((440741520.76003367 / (89)) + (89282664.26495636 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.058055470650726575 = (635797.9440763313 - 598886.3951942156) / 635797.9440763313 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[AppendAllLines - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_File.AppendAllLines.html>)	45.02 μs	41.22 μs	0.92	0.03	False
[AppendAllText - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_File.AppendAllText(size%3a%2010000).html>)	119.31 μs	77.38 μs	0.65	0.32	False
[ReadAllLines - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_File.ReadAllLines.html>)	175.16 μs	128.20 μs	0.73	0.05	False
[ReadAllLinesAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_File.ReadAllLinesAsync.html>)	426.57 μs	353.27 μs	0.83	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_File*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.Perf_File*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_File*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.Perf_File*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_File.AppendAllLines ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.76457367866698 (T) = (0 -40744.24383849207) / Math.Sqrt((3056004.4376065144 / (109)) + (127343.53112065271 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.15535321760942492 = (48238.204049241765 - 40744.24383849207) / 48238.204049241765 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllText(size: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.9868578232707 (T) = (0 -73550.43154166666) / Math.Sqrt((106535138.05701566 / (108)) + (4353408.880705925 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.4216783567178152 = (127179.10940396649 - 73550.43154166666) / 127179.10940396649 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.ReadAllLines ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 23.633768689124917 (T) = (0 -128491.76169537146) / Math.Sqrt((252372574.57318002 / (109)) + (72018.83561988953 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.21911630751483144 = (164546.60653297216 - 128491.76169537146) / 164546.60653297216 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.ReadAllLinesAsync ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 13.454123868325212 (T) = (0 -358623.4499142198) / Math.Sqrt((773435903.8126501 / (109)) + (50405571.38423705 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.1286806639948601 = (411586.6997264758 - 358623.4499142198) / 411586.6997264758 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).ExclusiveOrOperatorBenchmark.html>)	62.61 ns	11.22 ns	0.18	0.40	False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).EqualsAllBenchmark.html>)	56.72 ns	9.20 ns	0.16	0.34	False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).NegateBenchmark.html>)	261.80 ns	7.78 ns	0.03	0.31	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).BitwiseOrOperatorBenchmark.html>)	62.38 ns	11.22 ns	0.18	0.34	False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).AndNotBenchmark.html>)	67.54 ns	12.28 ns	0.18	0.38	False
[InequalityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).InequalityOperatorBenchmark.html>)	49.22 ns	8.87 ns	0.18	0.38	False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).MultiplyOperatorBenchmark.html>)	256.85 ns	12.23 ns	0.05	0.26	False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).SubtractBenchmark.html>)	259.31 ns	11.16 ns	0.04	0.23	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).BitwiseOrBenchmark.html>)	65.24 ns	12.64 ns	0.19	0.42	False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).ConditionalSelectBenchmark.html>)	91.16 ns	14.71 ns	0.16	0.23	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).XorBenchmark.html>)	62.44 ns	11.22 ns	0.18	0.30	False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).SubtractionOperatorBenchmark.html>)	256.72 ns	11.25 ns	0.04	0.18	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).OnesComplementBenchmark.html>)	43.70 ns	7.50 ns	0.17	0.45	False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).AddBenchmark.html>)	257.10 ns	11.15 ns	0.04	0.24	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).BitwiseAndBenchmark.html>)	63.51 ns	11.28 ns	0.18	0.34	False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).EqualityOperatorBenchmark.html>)	48.31 ns	9.30 ns	0.19	0.30	False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).UnaryNegateOperatorBenchmark.html>)	257.94 ns	7.66 ns	0.03	0.27	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).OnesComplementOperatorBenchmark.html>)	44.10 ns	7.39 ns	0.17	0.33	False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).MultiplyBenchmark.html>)	273.10 ns	10.88 ns	0.04	0.19	False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).EqualsStaticBenchmark.html>)	302.14 ns	11.09 ns	0.04	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.522892001760816 (T) = (0 -11.062948382832952) / Math.Sqrt((197.4871692351833 / (99)) + (0.03437847836407814 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8382109521208276 = (68.3788459593075 - 11.062948382832952) / 68.3788459593075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 103.26246934035812 (T) = (0 -9.342414169734019) / Math.Sqrt((19.250690892966468 / (99)) + (0.020703188155788234 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8309980960218616 = (55.279934425724164 - 9.342414169734019) / 55.279934425724164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 78.6552883802721 (T) = (0 -7.484461823233617) / Math.Sqrt((1112.3695671138612 / (99)) + (0.05356157373786582 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9724068516846436 = (271.243489060953 - 7.484461823233617) / 271.243489060953 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.65804088102057 (T) = (0 -10.962167852897307) / Math.Sqrt((182.80364017395164 / (99)) + (0.1878070244062769 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8389039540965311 = (68.04740483491443 - 10.962167852897307) / 68.04740483491443 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.16798787628582 (T) = (0 -11.366006840490334) / Math.Sqrt((177.09317781405096 / (99)) + (0.2155660012886278 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8368797870440721 = (69.67871506862993 - 11.366006840490334) / 69.67871506862993 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.InequalityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 66.0673538607568 (T) = (0 -9.442034587720752) / Math.Sqrt((28.973427105142022 / (99)) + (0.6187540304661884 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8148840031831391 = (51.00604350828714 - 9.442034587720752) / 51.00604350828714 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 79.87900606215554 (T) = (0 -11.654823327411394) / Math.Sqrt((1084.6109575374269 / (99)) + (0.4779443249188011 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9579261240120196 = (277.0085487427149 - 11.654823327411394) / 277.0085487427149 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 79.19674199781932 (T) = (0 -11.291947839740752) / Math.Sqrt((1125.7263391916404 / (98)) + (1.0207974622481972 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.9599135381617236 = (281.689810522482 - 11.291947839740752) / 281.689810522482 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 38.67699361046143 (T) = (0 -11.700033271005049) / Math.Sqrt((195.5139416704893 / (99)) + (0.9412466826962419 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8283712723001121 = (68.17059957155816 - 11.700033271005049) / 68.17059957155816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 55.98204369933196 (T) = (0 -14.713698185898997) / Math.Sqrt((205.02101022512775 / (99)) + (0.038008341060980125 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8457664833028983 = (95.39883743165328 - 14.713698185898997) / 95.39883743165328 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.45358795775257 (T) = (0 -11.021079316597351) / Math.Sqrt((173.36002914142324 / (99)) + (0.04124062851628285 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8362635620387651 = (67.30987588240207 - 11.021079316597351) / 67.30987588240207 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 81.58166718413594 (T) = (0 -11.120025568605483) / Math.Sqrt((1060.1553291862917 / (99)) + (0.04121345384323337 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9600249217692246 = (278.17395389221605 - 11.120025568605483) / 278.17395389221605 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.13813947890258 (T) = (0 -7.506629579554953) / Math.Sqrt((170.4318423632064 / (99)) + (0.020853656581258494 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8528981000986315 = (51.03013342851541 - 7.506629579554953) / 51.03013342851541 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 80.353513464474 (T) = (0 -11.026441221269977) / Math.Sqrt((1122.8778919639335 / (99)) + (0.022072211379593178 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9608556091189995 = (281.6863661205129 - 11.026441221269977) / 281.6863661205129 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.92687714465987 (T) = (0 -11.44924926105387) / Math.Sqrt((169.72403200921383 / (99)) + (0.6322298309021821 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8316482560503804 = (68.0079041205545 - 11.44924926105387) / 68.0079041205545 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 90.24533897504432 (T) = (0 -9.301800790428194) / Math.Sqrt((19.8062823921044 / (90)) + (0.011857561339231373 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.8205220942159935 = (51.826996475112054 - 9.301800790428194) / 51.826996475112054 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 75.09150079645634 (T) = (0 -7.626059911022185) / Math.Sqrt((1186.9414307214988 / (98)) + (0.02214006150234935 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.9716500972934495 = (268.9977454229552 - 7.626059911022185) / 268.9977454229552 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.63529863235763 (T) = (0 -7.5399135512444815) / Math.Sqrt((176.91893892174653 / (99)) + (0.03412840159245337 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8528408878688505 = (51.23647079716576 - 7.5399135512444815) / 51.23647079716576 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 82.24952314258235 (T) = (0 -11.214980111230117) / Math.Sqrt((1074.011895326489 / (99)) + (0.0791800305046001 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9602709679973452 = (282.2867697979823 - 11.214980111230117) / 282.2867697979823 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 88.70734680433803 (T) = (0 -11.09621973199526) / Math.Sqrt((1106.0489743060327 / (99)) + (0.03581391231751657 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9639356639437812 = (307.6784697962522 - 11.09621973199526) / 307.6784697962522 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).AndNotBenchmark.html>)	62.74 ns	10.98 ns	0.18	0.37	False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).EqualityOperatorBenchmark.html>)	50.19 ns	9.10 ns	0.18	0.31	False
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).ExclusiveOrOperatorBenchmark.html>)	79.28 ns	10.92 ns	0.14	0.30	False
[InequalityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).InequalityOperatorBenchmark.html>)	50.10 ns	9.13 ns	0.18	0.43	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).BitwiseAndBenchmark.html>)	62.27 ns	11.08 ns	0.18	0.38	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).XorBenchmark.html>)	61.84 ns	12.14 ns	0.20	0.40	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).BitwiseOrOperatorBenchmark.html>)	81.93 ns	11.01 ns	0.13	0.44	False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).EqualsBenchmark.html>)	61.59 ns	10.50 ns	0.17	0.29	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).BitwiseAndOperatorBenchmark.html>)	63.73 ns	12.51 ns	0.20	0.36	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).OnesComplementOperatorBenchmark.html>)	43.79 ns	7.88 ns	0.18	0.52	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).OnesComplementBenchmark.html>)	47.52 ns	7.64 ns	0.16	0.37	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 44.6505996645998 (T) = (0 -11.171921184724596) / Math.Sqrt((170.97256571705543 / (99)) + (0.011836131166915516 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8401341534189907 = (69.8831015107621 - 11.171921184724596) / 69.8831015107621 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 102.46721257503442 (T) = (0 -9.172815985825263) / Math.Sqrt((16.73883183600428 / (98)) + (0.00397819820358136 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.8222428652013236 = (51.603081902811866 - 9.172815985825263) / 51.603081902811866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.391180303472815 (T) = (0 -11.102008874422703) / Math.Sqrt((176.84818929273922 / (99)) + (0.010809840601552207 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8362250022663806 = (67.78817907529549 - 11.102008874422703) / 67.78817907529549 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.InequalityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 72.47646302236251 (T) = (0 -9.240526301338821) / Math.Sqrt((33.025890187802275 / (99)) + (0.00933929939814801 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8195167059958973 = (51.198790183476845 - 9.240526301338821) / 51.198790183476845 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.28435980683021 (T) = (0 -11.71387386089487) / Math.Sqrt((186.0661287389036 / (99)) + (1.0149165480410676 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8275886367795541 = (67.94142591354294 - 11.71387386089487) / 67.94142591354294 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 38.19743997671632 (T) = (0 -11.716394012331328) / Math.Sqrt((196.80170984222886 / (99)) + (0.901641180060032 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8266049687118082 = (67.5705291281274 - 11.716394012331328) / 67.5705291281274 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/3/2022 11:38:23 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.93913198041102 (T) = (0 -11.098913698527445) / Math.Sqrt((195.23690701143911 / (99)) + (0.016774098687252346 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8382811740271036 = (68.63093169120327 - 11.098913698527445) / 68.63093169120327 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 90.18539537178145 (T) = (0 -10.578323770922804) / Math.Sqrt((32.5748965225139 / (90)) + (0.012925135272666649 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.8372481294176568 = (64.996634036048 - 10.578323770922804) / 64.996634036048 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.770138715899385 (T) = (0 -11.756628845030546) / Math.Sqrt((174.98519715975664 / (99)) + (0.8428473920612566 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8272927904681046 = (68.07260030948126 - 11.756628845030546) / 68.07260030948126 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.01204788665543 (T) = (0 -7.6051885368884795) / Math.Sqrt((184.40267779384465 / (99)) + (0.07277550359651082 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8521454180791275 = (51.43694864294807 - 7.6051885368884795) / 51.43694864294807 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.76547136588259 (T) = (0 -7.512983991047573) / Math.Sqrt((173.2897110780689 / (99)) + (0.027238429558674136 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8524515923956434 = (50.9187737979067 - 7.512983991047573) / 50.9187737979067 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>)	2.14 μs	1.60 μs	0.75	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Globalization.Tests.StringSearch*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Globalization.Tests.StringSearch*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.9964084575012 (T) = (0 -1565.4511641342926) / Math.Sqrt((8681.993526679518 / (109)) + (1428.8856998631684 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.2670588755637819 = (2135.848449407782 - 1565.4511641342926) / 2135.848449407782 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Load - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName%3a%20%22deep.xml%22).html>)	1.51 ms	1.30 ms	0.86	0.03	False
[Load - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName%3a%20%22repeated.xml%22).html>)	1.95 ms	1.81 ms	0.93	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "deep.xml") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 1/3/2023 12:19:11 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 21.17323395219339 (T) = (0 -1316798.5200636617) / Math.Sqrt((5782879157.471118 / (109)) + (128193030.35042533 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.12181409285341879 = (1499453.0307850523 - 1316798.5200636617) / 1499453.0307850523 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.150050728942656 (T) = (0 -1834876.4965143837) / Math.Sqrt((2700983576.836433 / (98)) + (231497574.95166034 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.051101784383482396 = (1933691.5870605034 - 1834876.4965143837) / 1933691.5870605034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.Perf_Directory

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[GetCurrentDirectory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_Directory.GetCurrentDirectory.html>)	4.13 μs	2.78 μs	0.67	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Directory*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.Perf_Directory*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Directory*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.Perf_Directory*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_Directory.GetCurrentDirectory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.83794374854484 (T) = (0 -2783.196693220101) / Math.Sqrt((486606.6877551157 / (109)) + (621.2944819323452 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.23765965766028252 = (3650.8584665454327 - 2783.196693220101) / 3650.8584665454327 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ImmutableDictionary(String%2c%20String)).SerializeObjectProperty(Mode%3a%20Reflection).html>)	558.04 μs	513.54 μs	0.92	0.02	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ImmutableDictionary(String%2c%20String)).SerializeToString(Mode%3a%20SourceGen).html>)	509.19 μs	462.95 μs	0.91	0.03	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ImmutableDictionary(String%2c%20String)).SerializeToString(Mode%3a%20Reflection).html>)	557.62 μs	522.55 μs	0.94	0.03	False

Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 16.72632901813689 (T) = (0 -517114.896492889) / Math.Sqrt((298973263.55168027 / (89)) + (37236570.17141905 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.09094028722502426 = (568845.9066284605 - 517114.896492889) / 568845.9066284605 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 18.768626162904436 (T) = (0 -464835.7082546845) / Math.Sqrt((222873222.4409983 / (89)) + (31324795.753339835 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10089976488238725 = (517000.98620692565 - 464835.7082546845) / 517000.98620692565 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.551780813671765 (T) = (0 -509950.49038849253) / Math.Sqrt((228769809.01961553 / (89)) + (39943853.84663211 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.09465732409933185 = (563267.9249116088 - 509950.49038849253) / 563267.9249116088 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_ToString(IndexViewModel).SystemTextJsonSourceGen.html>)	742.49 μs	608.03 μs	0.82	0.03	False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_ToString(IndexViewModel).SystemTextJsonReflection.html>)	818.14 μs	692.94 μs	0.85	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.582414279513696 (T) = (0 -608207.4347619048) / Math.Sqrt((522503874.5049002 / (89)) + (61905650.5982646 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.17282069304819797 = (735278.8321109968 - 608207.4347619048) / 735278.8321109968 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 48.650820310562445 (T) = (0 -689589.8856725729) / Math.Sqrt((560629817.5773371 / (89)) + (6321848.170872868 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.1605846802039649 = (821512.1518631951 - 689589.8856725729) / 821512.1518631951 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[BitArrayAnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size%3a%20512).html>)	1.54 μs	1.06 μs	0.69	0.14	False
[BitArrayOr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Tests.Perf_BitArray.BitArrayOr(Size%3a%20512).html>)	1.55 μs	971.74 ns	0.63	0.09	False
[BitArrayNot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Tests.Perf_BitArray.BitArrayNot(Size%3a%20512).html>)	1.12 μs	699.07 ns	0.63	0.05	False
[BitArrayXor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Tests.Perf_BitArray.BitArrayXor(Size%3a%20512).html>)	1.54 μs	1.01 μs	0.66	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.Tests.Perf_BitArray*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.Tests.Perf_BitArray*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.21678390208405 (T) = (0 -1061.2873056027381) / Math.Sqrt((940.745775694993 / (109)) + (4780.119506408027 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.31523981719440297 = (1549.8671392580618 - 1061.2873056027381) / 1549.8671392580618 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.46052338036708 (T) = (0 -1002.3402851035971) / Math.Sqrt((977.0708434248028 / (109)) + (986.4841878118665 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3526036562987698 = (1548.263741146755 - 1002.3402851035971) / 1548.263741146755 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 27.433677118065177 (T) = (0 -722.4239511775604) / Math.Sqrt((219.48443610982744 / (109)) + (1245.177932598945 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.35471540332330576 = (1119.5431518095188 - 722.4239511775604) / 1119.5431518095188 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.335316831355094 (T) = (0 -1071.2601088149356) / Math.Sqrt((1607.2695202610228 / (108)) + (4474.471695634983 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.3085785255080338 = (1549.3590354596122 - 1071.2601088149356) / 1549.3590354596122 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>)	639.01 ms	287.89 ms	0.45	0.11	False
[ReadToEndAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>)	672.84 ms	334.80 ms	0.50	0.12	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>)	609.44 ms	267.25 ms	0.44	0.10	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>)	624.19 ms	268.52 ms	0.43	0.14	False
[ReadToEndAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>)	681.69 ms	338.83 ms	0.50	0.16	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>)	623.30 ms	284.34 ms	0.46	0.17	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>)	623.85 ms	269.52 ms	0.43	0.09	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>)	625.21 ms	265.37 ms	0.42	0.05	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%208%5d).html>)	624.67 ms	270.39 ms	0.43	0.13	False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>)	636.48 ms	266.65 ms	0.42	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 11.640292585176203 (T) = (0 -275443840.8225885) / Math.Sqrt((32568916113348816 / (109)) + (82679980550224.2 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4276403195986466 = (481242565.2160547 - 275443840.8225885) / 481242565.2160547 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.126432828021652 (T) = (0 -320837993.67777777) / Math.Sqrt((31416080775191908 / (108)) + (50084489088772.33 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.395337489445933 = (530606723.71400386 - 320837993.67777777) / 530606723.71400386 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.123526503497686 (T) = (0 -272453902.9595238) / Math.Sqrt((31895358749138756 / (109)) + (34269609658594.074 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4345718048775625 = (481854115.7122997 - 272453902.9595238) / 481854115.7122997 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.05169776091731 (T) = (0 -274070703.08174604) / Math.Sqrt((32108025745955452 / (100)) + (91481575343806.47 / (6))) is greater than 1.9830375264834914 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (100) + (6) - 2, .975) and 0.44642190810132376 = (495089504.24994487 - 274070703.08174604) / 495089504.24994487 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 11.826997136094452 (T) = (0 -324839695.2394994) / Math.Sqrt((30732517897550488 / (109)) + (86993856164637.89 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3853241937575106 = (528473208.05619967 - 324839695.2394994) / 528473208.05619967 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 11.835735829588085 (T) = (0 -273371372.03492063) / Math.Sqrt((32426132743226268 / (109)) + (78428253786567.62 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.43277916687812623 = (481948750.94826376 - 273371372.03492063) / 481948750.94826376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.299760485125734 (T) = (0 -269642739.46068376) / Math.Sqrt((32227110786439364 / (109)) + (2911220032991.087 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4397712399576004 = (481308277.426308 - 269642739.46068376) / 481308277.426308 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.320075768780441 (T) = (0 -268272083.2555556) / Math.Sqrt((32264751778530820 / (109)) + (6565849491621.785 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.44183079140541154 = (480628596.3552819 - 268272083.2555556) / 480628596.3552819 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 8]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.559691328722597 (T) = (0 -268755975.42539686) / Math.Sqrt((32460876783508056 / (102)) + (1788785886170.3699 / (6))) is greater than 1.9825972617651593 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (6) - 2, .975) and 0.4547655118391062 = (492918150.37586063 - 268755975.42539686) / 492918150.37586063 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.005576913244866 (T) = (0 -270506807.68333334) / Math.Sqrt((32556092306805588 / (109)) + (62708053192663.44 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.43830517545220926 = (481590350.954565 - 270506807.68333334) / 481590350.954565 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeObjectProperty(Mode%3a%20Reflection).html>)	14.56 μs	13.57 μs	0.93	0.04	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToString(Mode%3a%20Reflection).html>)	12.06 μs	11.04 μs	0.92	0.04	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToString(Mode%3a%20SourceGen).html>)	8.82 μs	8.28 μs	0.94	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 15.683199593408892 (T) = (0 -13621.835093601816) / Math.Sqrt((69053.27939685101 / (16)) + (1227.4992928286913 / (6))) is greater than 2.08596344725343 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (6) - 2, .975) and 0.07184685433899524 = (14676.279617520151 - 13621.835093601816) / 14676.279617520151 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 10.151543980580195 (T) = (0 -11149.704653716835) / Math.Sqrt((65510.30509410932 / (16)) + (29845.191158145542 / (6))) is greater than 2.08596344725343 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (6) - 2, .975) and 0.07978613973124901 = (12116.427642657472 - 11149.704653716835) / 12116.427642657472 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.01881877387964 (T) = (0 -8058.632024994302) / Math.Sqrt((75013.8850266778 / (16)) + (12548.741185989918 / (6))) is greater than 2.08596344725343 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (6) - 2, .975) and 0.10937197394605339 = (9048.257846431366 - 8058.632024994302) / 9048.257846431366 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToString(Mode%3a%20Reflection).html>)	473.15 μs	427.53 μs	0.90	0.03	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToString(Mode%3a%20SourceGen).html>)	450.64 μs	403.78 μs	0.90	0.03	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeObjectProperty(Mode%3a%20Reflection).html>)	475.55 μs	444.33 μs	0.93	0.03	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeObjectProperty(Mode%3a%20SourceGen).html>)	453.60 μs	409.69 μs	0.90	0.04	False

Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 23.582756753265585 (T) = (0 -426947.6526849996) / Math.Sqrt((201286459.04535925 / (89)) + (15673692.847592706 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10868970444423559 = (479011.2431258097 - 426947.6526849996) / 479011.2431258097 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.791122331995986 (T) = (0 -401992.4247354497) / Math.Sqrt((199273755.4649417 / (89)) + (3278848.460910908 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.11659972999232701 = (455051.28126342787 - 401992.4247354497) / 455051.28126342787 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.63036933158551 (T) = (0 -433024.9352069012) / Math.Sqrt((257350694.9346652 / (89)) + (32510104.5472807 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10503916477607107 = (483847.91620356624 - 433024.9352069012) / 483847.91620356624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 29.95330809725104 (T) = (0 -407778.68789925845) / Math.Sqrt((254696154.74570954 / (89)) + (1751722.896811638 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.11539345640547714 = (460971.819451261 - 407778.68789925845) / 460971.819451261 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).AndNotBenchmark.html>)	61.63 ns	12.81 ns	0.21	0.39	False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).EqualsBenchmark.html>)	59.19 ns	11.42 ns	0.19	0.43	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).BitwiseOrOperatorBenchmark.html>)	77.83 ns	11.92 ns	0.15	0.32	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).XorBenchmark.html>)	61.72 ns	10.99 ns	0.18	0.36	False
[InequalityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).InequalityOperatorBenchmark.html>)	47.14 ns	10.41 ns	0.22	0.42	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).BitwiseAndBenchmark.html>)	63.83 ns	11.41 ns	0.18	0.46	False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).EqualityOperatorBenchmark.html>)	47.71 ns	9.34 ns	0.20	0.41	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).OnesComplementOperatorBenchmark.html>)	47.89 ns	7.15 ns	0.15	0.49	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).OnesComplementBenchmark.html>)	47.76 ns	8.97 ns	0.19	0.60	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).BitwiseAndOperatorBenchmark.html>)	65.37 ns	11.13 ns	0.17	0.42	False
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).ExclusiveOrOperatorBenchmark.html>)	64.62 ns	10.93 ns	0.17	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.11344151123233 (T) = (0 -11.459276165289358) / Math.Sqrt((172.95086743683814 / (99)) + (0.6533168256017138 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8367456336538144 = (70.19276985823234 - 11.459276165289358) / 70.19276985823234 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 92.07596175177379 (T) = (0 -10.703165425672571) / Math.Sqrt((25.779552260829718 / (90)) + (0.4138661224350095 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.8368319784172323 = (65.5959747617786 - 10.703165425672571) / 65.5959747617786 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.66953384073871 (T) = (0 -11.495622415471578) / Math.Sqrt((179.78572023077612 / (99)) + (0.09918848351195568 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.830708637075301 = (67.90436450431817 - 11.495622415471578) / 67.90436450431817 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.82130926034886 (T) = (0 -11.517807756185206) / Math.Sqrt((179.91885302300796 / (99)) + (1.0309114332689864 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8298224231163127 = (67.68111267712656 - 11.517807756185206) / 67.68111267712656 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.InequalityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 64.6651956201301 (T) = (0 -9.931033820625208) / Math.Sqrt((33.34069045129278 / (99)) + (0.44249299029433187 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8066514994819611 = (51.36338680681246 - 9.931033820625208) / 51.36338680681246 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.64653179124549 (T) = (0 -11.730566686410834) / Math.Sqrt((189.40340018274065 / (99)) + (0.6824734976863136 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8314546747006485 = (69.59888484344671 - 11.730566686410834) / 69.59888484344671 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 59.94876119286645 (T) = (0 -10.014970201122788) / Math.Sqrt((23.087152431668827 / (98)) + (1.4895065422971983 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.8063402127741125 = (51.71424767415024 - 10.014970201122788) / 51.71424767415024 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.378415390854904 (T) = (0 -7.6855873376626525) / Math.Sqrt((164.89570120229652 / (99)) + (0.3395228469967345 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8507332942038243 = (51.488959287125645 - 7.6855873376626525) / 51.488959287125645 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.743746572006696 (T) = (0 -7.972074737413696) / Math.Sqrt((181.0472662121846 / (99)) + (0.5275767189254472 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8464533637193674 = (51.919566136527905 - 7.972074737413696) / 51.919566136527905 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.851998393143205 (T) = (0 -11.305945189447172) / Math.Sqrt((173.42393672096975 / (99)) + (0.11360232614058977 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8345340907572493 = (68.32794284447147 - 11.305945189447172) / 68.32794284447147 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.95862828155149 (T) = (0 -11.290418486182917) / Math.Sqrt((172.7360781710364 / (99)) + (0.32520814699410094 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8329037322000795 = (67.56834628827232 - 11.290418486182917) / 67.56834628827232 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).ExclusiveOrOperatorBenchmark.html>)	60.73 ns	12.20 ns	0.20	0.37	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).BitwiseOrOperatorBenchmark.html>)	65.09 ns	12.87 ns	0.20	0.41	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).OnesComplementBenchmark.html>)	46.14 ns	7.80 ns	0.17	0.43	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).OnesComplementOperatorBenchmark.html>)	43.67 ns	7.33 ns	0.17	0.49	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).BitwiseAndOperatorBenchmark.html>)	62.10 ns	10.89 ns	0.18	0.39	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.69462577340173 (T) = (0 -11.481982308366469) / Math.Sqrt((172.7225214779986 / (99)) + (0.31002367253020424 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.829553979465593 = (67.36433195897735 - 11.481982308366469) / 67.36433195897735 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.73401559310126 (T) = (0 -11.913851608791447) / Math.Sqrt((177.97858767776637 / (99)) + (0.8870584284434597 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8230708349399455 = (67.33684412486524 - 11.913851608791447) / 67.33684412486524 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.07412573169258 (T) = (0 -7.931342715403789) / Math.Sqrt((159.21765472637043 / (99)) + (0.43635807179183345 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8439107843066551 = (50.8128808269869 - 7.931342715403789) / 50.8128808269869 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.469903670685156 (T) = (0 -7.517610835404252) / Math.Sqrt((179.70545265081284 / (99)) + (0.05460179998232874 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8536681197049195 = (51.373704897694736 - 7.517610835404252) / 51.373704897694736 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.441408097024855 (T) = (0 -11.82785619676262) / Math.Sqrt((171.1595578754425 / (99)) + (0.40510785066180766 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8278634326336033 = (68.71204868159562 - 11.82785619676262) / 68.71204868159562 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.XmlFromStream(CollectionsOfPrimitives).XmlSerializer.html>)	11.22 ms	10.36 ms	0.92	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 9.700906087762492 (T) = (0 -10539852.002803659) / Math.Sqrt((93200470312.96794 / (109)) + (18636646688.13075 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.05475612908539181 = (11150405.019400345 - 10539852.002803659) / 11150405.019400345 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in BenchmarksGame.ReverseComplement_6

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[RunBench - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/BenchmarksGame.ReverseComplement_6.RunBench.html>)	14.28 ms	10.17 ms	0.71	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'BenchmarksGame.ReverseComplement_6*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'BenchmarksGame.ReverseComplement_6*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'BenchmarksGame.ReverseComplement_6*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'BenchmarksGame.ReverseComplement_6*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### BenchmarksGame.ReverseComplement_6.RunBench ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 35.884130739993765 (T) = (0 -10257409.395528955) / Math.Sqrt((363155373128.11755 / (101)) + (52391678205.73126 / (6))) is greater than 1.9828152737947098 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (101) + (6) - 2, .975) and 0.2797575138604117 = (14241605.560520897 - 10257409.395528955) / 14241605.560520897 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.JsonToStream(IndexViewModel).JsonNet.html>)	700.19 μs	622.86 μs	0.89	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.787361942348454 (T) = (0 -638959.5439020704) / Math.Sqrt((724251469.5442733 / (107)) + (209783977.70125455 / (6))) is greater than 1.9815667570746824 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (107) + (6) - 2, .975) and 0.11448340807512308 = (721566.9923396271 - 638959.5439020704) / 721566.9923396271 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[TryWriteBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address%3a%201020%3a3040%3a5060%3a7080%3a9010%3a1112%3a1314%3a1516).html>)	569.55 ns	78.54 ns	0.14	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 149.5792711324893 (T) = (0 -80.6421422171142) / Math.Sqrt((772.4122055391231 / (100)) + (8.403172702732027 / (6))) is greater than 1.9830375264834914 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (100) + (6) - 2, .975) and 0.8485527590372793 = (532.4767998709505 - 80.6421422171142) / 532.4767998709505 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_FromString(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>)	6.60 ms	6.19 ms	0.94	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 16.26918235514919 (T) = (0 -6191322.1245802445) / Math.Sqrt((14462711183.848114 / (89)) + (2349655928.314376 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.058252612690949473 = (6574291.798431564 - 6191322.1245802445) / 6574291.798431564 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20SourceGen).html>)	968.72 μs	910.42 μs	0.94	0.03	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20SourceGen).html>)	996.64 μs	927.05 μs	0.93	0.02	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>)	1.00 ms	930.19 μs	0.93	0.03	False

Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 18.041284647063847 (T) = (0 -916853.8885510935) / Math.Sqrt((500663616.93650436 / (89)) + (13039281.999953564 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.05208883146454923 = (967236.0860223415 - 916853.8885510935) / 967236.0860223415 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 15.860430468663282 (T) = (0 -930221.3461420626) / Math.Sqrt((370691213.1294784 / (89)) + (106347387.36054114 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07387815659696739 = (1004426.5263455685 - 930221.3461420626) / 1004426.5263455685 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 22.50496381082304 (T) = (0 -929471.2412983463) / Math.Sqrt((410439909.71599805 / (89)) + (34934721.026406236 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07253818517464336 = (1002166.5867433777 - 929471.2412983463) / 1002166.5867433777 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).ConditionalSelectBenchmark.html>)	89.77 ns	15.27 ns	0.17	0.26	False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).AddBenchmark.html>)	406.43 ns	10.62 ns	0.03	0.21	False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).EqualsStaticBenchmark.html>)	451.31 ns	11.43 ns	0.03	0.22	False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).AndNotBenchmark.html>)	65.47 ns	11.50 ns	0.18	0.40	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).OnesComplementOperatorBenchmark.html>)	43.22 ns	8.85 ns	0.20	0.47	False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).UnaryNegateOperatorBenchmark.html>)	429.40 ns	7.69 ns	0.02	0.26	False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).SubtractBenchmark.html>)	461.22 ns	11.90 ns	0.03	0.20	False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).EqualsAllBenchmark.html>)	55.08 ns	9.61 ns	0.17	0.37	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).BitwiseAndBenchmark.html>)	65.55 ns	11.02 ns	0.17	0.35	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).BitwiseOrBenchmark.html>)	64.23 ns	11.14 ns	0.17	0.41	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).XorBenchmark.html>)	61.40 ns	11.87 ns	0.19	0.35	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).OnesComplementBenchmark.html>)	55.68 ns	7.58 ns	0.14	0.46	False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).SubtractionOperatorBenchmark.html>)	419.69 ns	11.26 ns	0.03	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 53.63770315034236 (T) = (0 -15.48929369427085) / Math.Sqrt((187.45261511358248 / (98)) + (1.5697507485645503 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.8362355413134266 = (94.58275512585799 - 15.48929369427085) / 94.58275512585799 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 225.84181629995342 (T) = (0 -11.30751310196692) / Math.Sqrt((341.6299794200777 / (99)) + (0.2955142607005624 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9739351621744681 = (433.8224997851555 - 11.30751310196692) / 433.8224997851555 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 181.18798664418554 (T) = (0 -11.184178300244204) / Math.Sqrt((619.9949865843209 / (99)) + (0.07297639540082072 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9759505750364189 = (465.04971811928056 - 11.184178300244204) / 465.04971811928056 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 44.389894415670526 (T) = (0 -11.562994941349494) / Math.Sqrt((159.40690277992388 / (99)) + (0.7960828384405751 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8352033765456669 = (70.16524185371871 - 11.562994941349494) / 70.16524185371871 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 30.82369283716853 (T) = (0 -7.855159584462673) / Math.Sqrt((192.46633458440166 / (99)) + (0.49332377039426323 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8481578106888099 = (51.73239150526253 - 7.855159584462673) / 51.73239150526253 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 212.25252316158262 (T) = (0 -7.777372095197183) / Math.Sqrt((334.652660237679 / (90)) + (0.3163250480527072 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.9814805273540796 = (419.9564557746978 - 7.777372095197183) / 419.9564557746978 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 240.69958343626476 (T) = (0 -11.543076051420401) / Math.Sqrt((274.3811416730374 / (90)) + (0.13670339586386074 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.9733652040756469 = (433.38331122207717 - 11.543076051420401) / 433.38331122207717 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 92.62408461410739 (T) = (0 -9.747950645723407) / Math.Sqrt((21.899310577288077 / (99)) + (0.09831705702145851 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8224288417963658 = (54.89602446893259 - 9.747950645723407) / 54.89602446893259 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.8885482800187 (T) = (0 -11.759196172728643) / Math.Sqrt((167.27923805767983 / (99)) + (1.1395030824553765 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8266037002940584 = (67.81688071008877 - 11.759196172728643) / 67.81688071008877 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 38.50468861166305 (T) = (0 -11.820339365381146) / Math.Sqrt((202.4160789217082 / (99)) + (0.4657579114868155 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8259501621931051 = (67.91353278074061 - 11.820339365381146) / 67.91353278074061 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.75666566005778 (T) = (0 -11.371915684903586) / Math.Sqrt((178.7658663863254 / (99)) + (0.17966071613243234 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8326344214381814 = (67.9465621463091 - 11.371915684903586) / 67.9465621463091 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.5405851843065 (T) = (0 -7.495447256530213) / Math.Sqrt((183.96473635852394 / (99)) + (0.02266776162569742 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8555758033388005 = (51.89883295050318 - 7.495447256530213) / 51.89883295050318 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 229.40456570982525 (T) = (0 -11.245423374786531) / Math.Sqrt((306.9572943154058 / (92)) + (0.21222606375573477 / (6))) is greater than 1.9849843115220658 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (92) + (6) - 2, .975) and 0.9739985783411969 = (432.49263530093293 - 11.245423374786531) / 432.49263530093293 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[IndexOfAny - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.IndexOfAny.html>)	622.46 ns	418.07 ns	0.67	0.21	False
[Split - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.Split(s%3a%20%22ABCDEFGHIJKLMNOPQRSTUVWXYZ%22%2c%20arr%3a%20%5b%27%20%27%5d%2c%20options%3a%20None).html>)	758.68 ns	585.21 ns	0.77	0.09	False
[Replace_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.Replace_String(text%3a%20%22This%20is%20a%20very%20nice%20sentence.%20This%20is%20another%20very%20nice%20sentence.%22%2c%20oldValue%3a%20%22a%22%2c%20newValue%3a%20%22b%22).html>)	6.24 μs	628.63 ns	0.10	0.06	False
[Replace_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.Replace_Char(text%3a%20%22This%20is%20a%20very%20nice%20sentence%22%2c%20oldChar%3a%20%27i%27%2c%20newChar%3a%20%27I%27).html>)	2.96 μs	401.87 ns	0.14	0.07	False
[Replace_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.Replace_Char(text%3a%20%22yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l%22%2c%20oldChar%3a%20%27b%27%2c%20newChar%3a%20%27%2b%27).html>)	9.06 μs	625.67 ns	0.07	0.07	False
[Replace_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_String.Replace_Char(text%3a%20%22yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi79.html>)	100.35 μs	3.39 μs	0.03	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.IndexOfAny ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.94226726279577 (T) = (0 -405.73739708030536) / Math.Sqrt((965.037805435065 / (109)) + (118.786671010087 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.30293729168813993 = (582.0672835345279 - 405.73739708030536) / 582.0672835345279 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 25.335350600894916 (T) = (0 -594.2392242654395) / Math.Sqrt((1765.6717689013642 / (109)) + (61.50658634060228 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.17983663301400493 = (724.5376326050741 - 594.2392242654395) / 724.5376326050741 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 11/9/2022 1:30:37 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.026409499129024 (T) = (0 -639.1345741264005) / Math.Sqrt((1927750.2914931953 / (109)) + (89.86249835361971 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.8928420577293215 = (5964.416268016426 - 639.1345741264005) / 5964.416268016426 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 11/9/2022 1:30:37 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.8296987292001 (T) = (0 -402.403145276744) / Math.Sqrt((389476.6052622854 / (109)) + (24.057515017533504 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.8554898830515266 = (2784.6018934454605 - 402.403145276744) / 2784.6018934454605 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 11/9/2022 1:30:37 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.76726140011915 (T) = (0 -629.7826195255576) / Math.Sqrt((3952755.1114664404 / (109)) + (69.02318417528049 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.9249753924230615 = (8394.347399680946 - 629.7826195255576) / 8394.347399680946 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 11/9/2022 1:30:37 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.2557146147606 (T) = (0 -3325.7331207323646) / Math.Sqrt((570758640.7654835 / (109)) + (19383.138572780637 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.9651651022333828 = (95471.304179324 - 3325.7331207323646) / 95471.304179324 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Hashtable>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromString(Mode%3a%20Reflection).html>)	588.83 μs	519.52 μs	0.88	0.02	False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromReader(Mode%3a%20Reflection).html>)	697.19 μs	610.93 μs	0.88	0.03	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromString(Mode%3a%20SourceGen).html>)	588.75 μs	523.76 μs	0.89	0.02	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>)	533.15 μs	491.60 μs	0.92	0.03	False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromStream(Mode%3a%20SourceGen).html>)	548.01 μs	508.65 μs	0.93	0.02	False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromReader(Mode%3a%20SourceGen).html>)	673.29 μs	614.20 μs	0.91	0.02	False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromStream(Mode%3a%20Reflection).html>)	549.42 μs	503.89 μs	0.92	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<Hashtable>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<Hashtable>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<Hashtable>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<Hashtable>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 26.555207094385505 (T) = (0 -524970.6005487926) / Math.Sqrt((103007342.05823441 / (89)) + (35905250.22007639 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.11908240087806814 = (595936.1023914894 - 524970.6005487926) / 595936.1023914894 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 22.216596285289484 (T) = (0 -617151.1130990243) / Math.Sqrt((180020773.1701634 / (89)) + (33516026.102528572 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.09032874652837061 = (678433.1270706378 - 617151.1130990243) / 678433.1270706378 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 55.54177799066832 (T) = (0 -522779.7506241733) / Math.Sqrt((97973514.8705812 / (89)) + (3957331.3695294545 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.12354720301540786 = (596472.2257978757 - 522779.7506241733) / 596472.2257978757 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 18.957961137006773 (T) = (0 -496494.9626767614) / Math.Sqrt((133511443.0107196 / (89)) + (17903648.346069843 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07480748955070926 = (536639.62588246 - 496494.9626767614) / 536639.62588246 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.924803906326176 (T) = (0 -509846.38775006117) / Math.Sqrt((93450669.58061479 / (89)) + (3832638.69416555 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07958788209018798 = (553932.7197341607 - 509846.38775006117) / 553932.7197341607 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 10.448817065290026 (T) = (0 -621931.992436866) / Math.Sqrt((132602561.8580844 / (89)) + (163479936.507275 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.08262101029236335 = (677944.4476214483 - 621931.992436866) / 677944.4476214483 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 19.97075390462342 (T) = (0 -509731.50211336603) / Math.Sqrt((131984437.83777577 / (89)) + (21270270.076142002 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.0807572521250231 = (554512.3997896287 - 509731.50211336603) / 554512.3997896287 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonReflection.html>)	4.43 ms	3.90 ms	0.88	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 15.334001452022132 (T) = (0 -3887279.6692708335) / Math.Sqrt((83866416291.12737 / (89)) + (147161849.88427958 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10925511104580536 = (4364077.433927024 - 3887279.6692708335) / 4364077.433927024 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToString(Mode%3a%20Reflection).html>)	248.27 μs	225.65 μs	0.91	0.03	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeObjectProperty(Mode%3a%20SourceGen).html>)	239.16 μs	213.68 μs	0.89	0.03	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeObjectProperty(Mode%3a%20Reflection).html>)	253.86 μs	228.55 μs	0.90	0.03	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToString(Mode%3a%20SourceGen).html>)	232.62 μs	208.70 μs	0.90	0.03	False

Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 19.029960073450354 (T) = (0 -224693.61030792582) / Math.Sqrt((53155599.14466803 / (89)) + (7513328.034697572 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.10328256861270832 = (250573.48328816058 - 224693.61030792582) / 250573.48328816058 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 35.58934211533524 (T) = (0 -212908.21089093908) / Math.Sqrt((55058839.99652414 / (89)) + (443714.68130461127 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.12212354523072647 = (242526.3939296519 - 212908.21089093908) / 242526.3939296519 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.757270726621844 (T) = (0 -228208.6199651924) / Math.Sqrt((54050606.8552857 / (89)) + (1310372.5187342414 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.1122564421282455 = (257065.92623695495 - 228208.6199651924) / 257065.92623695495 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 30.666056843795506 (T) = (0 -208504.2087587719) / Math.Sqrt((53021578.735908635 / (89)) + (1049478.472124636 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.11435018722536415 = (235425.1147025628 - 208504.2087587719) / 235425.1147025628 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[WriteAsciiCharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars%3a%208000).html>)	53.43 μs	32.14 μs	0.60	0.42	False
[WriteAsciiString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars%3a%208000).html>)	55.28 μs	30.28 μs	0.55	0.42	False
[WriteAsciiString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars%3a%202000000).html>)	23.98 ms	15.41 ms	0.64	0.23	False
[WriteAsciiCharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars%3a%202000000).html>)	15.82 ms	8.04 ms	0.51	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.786182781371394 (T) = (0 -31567.122786902506) / Math.Sqrt((90512658.17896947 / (109)) + (1059579.3939515292 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.5103417299903353 = (64467.65983607189 - 31567.122786902506) / 64467.65983607189 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 29.243246117001874 (T) = (0 -31222.140844712136) / Math.Sqrt((102430191.73213941 / (109)) + (2529341.8580897474 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.5221692463157662 = (65341.421840220755 - 31222.140844712136) / 65341.421840220755 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 52.309796615985675 (T) = (0 -15745292.981415832) / Math.Sqrt((4069835210673.081 / (109)) + (30316253649.641926 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4061819946214876 = (26515351.23354746 - 15745292.981415832) / 26515351.23354746 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 60.07462310624318 (T) = (0 -8011053.490671742) / Math.Sqrt((3080955328390.025 / (109)) + (11010282060.668827 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.5654133429694176 = (18433730.905152928 - 8011053.490671742) / 18433730.905152928 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrOperatorBenchmark.html>)	61.81 ns	12.22 ns	0.20	0.39	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseAndOperatorBenchmark.html>)	62.40 ns	11.19 ns	0.18	0.36	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrBenchmark.html>)	59.90 ns	10.89 ns	0.18	0.33	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseAndBenchmark.html>)	62.19 ns	10.85 ns	0.17	0.38	False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark.html>)	91.30 ns	14.63 ns	0.16	0.28	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.29750815326799 (T) = (0 -11.586291684325724) / Math.Sqrt((181.52759387776803 / (99)) + (0.9537785343133743 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8307821839631473 = (68.46969164170297 - 11.586291684325724) / 68.46969164170297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.39606823008167 (T) = (0 -11.386281048218246) / Math.Sqrt((192.32714028835957 / (99)) + (0.7946259510429574 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8328929035950888 = (68.13762726526322 - 11.386281048218246) / 68.13762726526322 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 46.76049948661194 (T) = (0 -11.302204911230342) / Math.Sqrt((118.97764276922318 / (92)) + (0.31204354071687923 / (6))) is greater than 1.9849843115220658 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (92) + (6) - 2, .975) and 0.8275451401485672 = (65.53717837216655 - 11.302204911230342) / 65.53717837216655 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.33104487262801 (T) = (0 -11.460101301692065) / Math.Sqrt((176.41666776993944 / (99)) + (0.7608015264214348 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8328519970232384 = (68.5625978031299 - 11.460101301692065) / 68.5625978031299 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 62.90406814402295 (T) = (0 -15.124897320615942) / Math.Sqrt((115.83663354804833 / (90)) + (1.3723108834937245 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.8366126915734922 = (92.5708212362111 - 15.124897320615942) / 92.5708212362111 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[GetFullPathWithRedundantSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.Perf_Path.GetFullPathWithRedundantSegments.html>)	19.36 μs	17.07 μs	0.88	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Path*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.Perf_Path*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Path*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.Perf_Path*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_Path.GetFullPathWithRedundantSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 24.31068548543662 (T) = (0 -17070.158165497527) / Math.Sqrt((370690.15372156963 / (103)) + (50493.69996839699 / (6))) is greater than 1.9823833701754254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (103) + (6) - 2, .975) and 0.13502574953653346 = (19734.87436920934 - 17070.158165497527) / 19734.87436920934 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AndNotBenchmark.html>)	63.50 ns	11.13 ns	0.18	0.36	False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyBenchmark.html>)	190.31 ns	11.14 ns	0.06	0.23	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseAndOperatorBenchmark.html>)	62.10 ns	11.32 ns	0.18	0.41	False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractionOperatorBenchmark.html>)	171.24 ns	11.04 ns	0.06	0.26	False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsBenchmark.html>)	61.44 ns	10.33 ns	0.17	0.36	False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddBenchmark.html>)	165.69 ns	11.28 ns	0.07	0.19	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementOperatorBenchmark.html>)	55.70 ns	7.42 ns	0.13	0.49	False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractBenchmark.html>)	162.93 ns	11.14 ns	0.07	0.31	False
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ExclusiveOrOperatorBenchmark.html>)	62.36 ns	11.29 ns	0.18	0.35	False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.NegateBenchmark.html>)	161.27 ns	7.75 ns	0.05	0.21	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrOperatorBenchmark.html>)	78.04 ns	10.88 ns	0.14	0.32	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementBenchmark.html>)	47.09 ns	7.39 ns	0.16	0.50	False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyOperatorBenchmark.html>)	171.31 ns	10.96 ns	0.06	0.28	False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualityOperatorBenchmark.html>)	48.90 ns	9.28 ns	0.19	0.43	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.XorBenchmark.html>)	59.74 ns	10.78 ns	0.18	0.38	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseAndBenchmark.html>)	58.13 ns	11.23 ns	0.19	0.40	False
[InequalityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.InequalityOperatorBenchmark.html>)	46.54 ns	9.23 ns	0.20	0.43	False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.UnaryNegateOperatorBenchmark.html>)	159.67 ns	7.55 ns	0.05	0.35	False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddOperatorBenchmark.html>)	193.71 ns	12.40 ns	0.06	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/3/2022 11:38:23 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.75637156024062 (T) = (0 -11.028630837126146) / Math.Sqrt((195.63887555641728 / (99)) + (0.014370843438423531 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8450383698633234 = (71.17007498823327 - 11.028630837126146) / 71.17007498823327 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 86.43673424265235 (T) = (0 -11.808914994959524) / Math.Sqrt((376.3877974370963 / (99)) + (0.8318159079736909 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9356085301650156 = (183.39253670124575 - 11.808914994959524) / 183.39253670124575 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.891370538132556 (T) = (0 -11.201200540073474) / Math.Sqrt((207.91118530820583 / (99)) + (0.027094048412715243 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8378352353596468 = (69.07296147171884 - 11.201200540073474) / 69.07296147171884 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 90.21597195367498 (T) = (0 -11.037647674936695) / Math.Sqrt((350.11770050430715 / (99)) + (0.022726292456102644 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9389463026682805 = (180.7859008925617 - 11.037647674936695) / 180.7859008925617 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 88.50941517330601 (T) = (0 -10.50951847289617) / Math.Sqrt((33.84849677892461 / (90)) + (0.05090760493544301 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.8392994084310924 = (65.39813183195248 - 10.50951847289617) / 65.39813183195248 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 82.86144586366046 (T) = (0 -12.009821540582129) / Math.Sqrt((393.9097812313969 / (99)) + (1.6731217795407833 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9343684546600095 = (182.988553421495 - 12.009821540582129) / 182.988553421495 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.6000412412601 (T) = (0 -7.596316699646363) / Math.Sqrt((179.94131770199698 / (99)) + (0.01425502579624428 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8527153498973283 = (51.575752764127145 - 7.596316699646363) / 51.575752764127145 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 86.23409752530553 (T) = (0 -11.049341643110518) / Math.Sqrt((401.98466404868 / (99)) + (0.02281280851202335 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9402406299530376 = (184.89722422487569 - 11.049341643110518) / 184.89722422487569 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.32604645244054 (T) = (0 -11.122401312517058) / Math.Sqrt((176.46809874005484 / (99)) + (0.020206592001493173 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8356752399035132 = (67.68548638675209 - 11.122401312517058) / 67.68548638675209 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 83.9569718413585 (T) = (0 -7.506406417079617) / Math.Sqrt((383.80067028294894 / (99)) + (0.025313689402383162 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9565861840499773 = (172.90363108649285 - 7.506406417079617) / 172.90363108649285 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.32869998221179 (T) = (0 -11.010173790691004) / Math.Sqrt((187.58635738086488 / (99)) + (0.028432713971590706 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8380168746014953 = (67.97111590237685 - 11.010173790691004) / 67.97111590237685 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.986925041252235 (T) = (0 -7.412145596862225) / Math.Sqrt((173.606357878002 / (99)) + (0.021006881497971593 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8550565433558288 = (51.138187045301834 - 7.412145596862225) / 51.138187045301834 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 81.94442344858284 (T) = (0 -11.182862547432537) / Math.Sqrt((424.8172192378537 / (99)) + (0.06093905592642177 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9382608876313698 = (181.13092524981903 - 11.182862547432537) / 181.13092524981903 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 90.73969945752015 (T) = (0 -9.281995895487896) / Math.Sqrt((21.49131148834626 / (98)) + (0.03347356639969336 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.8225645085566807 = (52.31194627402361 - 9.281995895487896) / 52.31194627402361 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.36760462399945 (T) = (0 -11.011038464162496) / Math.Sqrt((186.46620936613084 / (99)) + (0.05223593132318436 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8378708462863989 = (67.91522814961051 - 11.011038464162496) / 67.91522814961051 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.84128238381385 (T) = (0 -11.543437937023294) / Math.Sqrt((188.84115916890124 / (99)) + (0.7237730043162 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8309450637928478 = (68.28217025782965 - 11.543437937023294) / 68.28217025782965 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.InequalityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 76.89257244057006 (T) = (0 -9.259608900014937) / Math.Sqrt((28.66399821320982 / (99)) + (0.0102686086351388 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8175678078432169 = (50.75644156080297 - 9.259608900014937) / 50.75644156080297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 78.48666956029705 (T) = (0 -7.706859078374726) / Math.Sqrt((422.453305074888 / (99)) + (0.3949132396333267 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.954952792507431 = (171.08405842129184 - 7.706859078374726) / 171.08405842129184 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 83.75660968398918 (T) = (0 -11.81271074023032) / Math.Sqrt((382.58611678598413 / (99)) + (0.8871955163730101 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9342220987589112 = (179.58479242039724 - 11.81271074023032) / 179.58479242039724 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20SourceGen).html>)	246.66 μs	208.45 μs	0.85	0.03	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>)	221.78 μs	195.13 μs	0.88	0.03	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20Reflection).html>)	246.71 μs	215.43 μs	0.87	0.02	False

Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 34.40286084184879 (T) = (0 -209940.2820100309) / Math.Sqrt((82167673.95955402 / (89)) + (896787.9742712888 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.14509577587939393 = (245571.6980764543 - 209940.2820100309) / 245571.6980764543 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 14.745805893195215 (T) = (0 -194800.7439588727) / Math.Sqrt((143118741.01610237 / (89)) + (1552347.847447059 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.09373114014194145 = (214948.07179999846 - 194800.7439588727) / 214948.07179999846 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 21.64520426130609 (T) = (0 -211499.6458470916) / Math.Sqrt((80971440.32405281 / (89)) + (9260867.801890466 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.13815149811459582 = (245402.34784235162 - 211499.6458470916) / 245402.34784235162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%22%2c%20Options%3a%20None).html>)	7.34 ms	4.30 ms	0.59	0.22	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%22%2c%20Options%3a%20NonBacktracking).html>)	6.89 ms	4.48 ms	0.65	0.19	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20Compiled).html>)	7.22 ms	4.86 ms	0.67	0.15	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%5c%5cs%2b%5c%5cw%2b%22%2c%20Options%3a%20Compiled).html>)	24.77 ms	23.04 ms	0.93	0.01	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20None).html>)	9.18 ms	6.58 ms	0.72	0.14	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%5c%5cs%2b%5c%5cw%2b%22%2c%20Options%3a%20None).html>)	25.20 ms	22.33 ms	0.89	0.02	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%7cWatson%22%2c%20Options%3a%20Compiled).html>)	8.48 ms	5.30 ms	0.62	0.21	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes.%7b0%2c25%7dWatson%7cWatson.%7b0%2c25%7dHolmes%22%2c%20Options%3a%20None).html>)	12.49 ms	9.64 ms	0.77	0.07	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes.%7b0%2c25%7dWatson%7cWatson.%7b0%2c25%7dHolmes%22%2c%20Options%3a%20NonBacktracking).html>)	7.11 ms	4.61 ms	0.65	0.16	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes.%7b0%2c25%7dWatson%7cWatson.%7b0%2c25%7dHolmes%22%2c%20Options%3a%20Compiled).html>)	10.49 ms	7.94 ms	0.76	0.09	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%22%2c%20Options%3a%20NonBacktracking).html>)	23.89 ms	21.27 ms	0.89	0.01	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%7cWatson%22%2c%20Options%3a%20NonBacktracking).html>)	8.61 ms	5.42 ms	0.63	0.20	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%22%2c%20Options%3a%20Compiled).html>)	6.66 ms	4.16 ms	0.62	0.18	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%7cWatson%22%2c%20Options%3a%20None).html>)	12.24 ms	9.18 ms	0.75	0.10	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%22%2c%20Options%3a%20Compiled).html>)	16.70 ms	14.10 ms	0.84	0.02	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%22%2c%20Options%3a%20Compiled).html>)	7.33 ms	4.61 ms	0.63	0.25	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%22%2c%20Options%3a%20NonBacktracking).html>)	7.31 ms	4.28 ms	0.58	0.21	False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20NonBacktracking).html>)	7.53 ms	4.97 ms	0.66	0.17	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 74.76147488198862 (T) = (0 -4274725.7996279765) / Math.Sqrt((138792143376.00815 / (108)) + (1056089431.7835646 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.40066603193544426 = (7132460.41006562 - 4274725.7996279765) / 7132460.41006562 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 72.38825740546316 (T) = (0 -4404203.569928164) / Math.Sqrt((71755665444.74855 / (109)) + (2477908171.3772535 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.34979064836013973 = (6773516.189548095 - 4404203.569928164) / 6773516.189548095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 36.43852651249812 (T) = (0 -4798664.16400145) / Math.Sqrt((80360263171.25722 / (109)) + (18684915212.32785 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.32030513530189103 = (7060027.09926727 - 4798664.16400145) / 7060027.09926727 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\\s+\\w+", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/26/2022 8:24:36 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.47646842386191 (T) = (0 -22747785.551060054) / Math.Sqrt((381347365191.54675 / (105)) + (83186677727.1038 / (6))) is greater than 1.9819674897361885 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (6) - 2, .975) and 0.09224766007938746 = (25059462.312208924 - 22747785.551060054) / 25059462.312208924 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.021172514694605 (T) = (0 -6676822.420557804) / Math.Sqrt((100464697623.64671 / (109)) + (15380024360.461573 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.2565120626214106 = (8980404.502726879 - 6676822.420557804) / 8980404.502726879 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\\s+\\w+", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/25/2022 11:45:49 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 11.7862669783192 (T) = (0 -22711971.08189033) / Math.Sqrt((392943220258.9158 / (105)) + (213718042633.31055 / (6))) is greater than 1.9819674897361885 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (6) - 2, .975) and 0.09334713832777368 = (25050349.524073057 - 22711971.08189033) / 25050349.524073057 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 56.69297284672361 (T) = (0 -5406526.632638889) / Math.Sqrt((122048869184.39806 / (109)) + (8894745121.41042 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.34849504111255314 = (8298519.541388347 - 5406526.632638889) / 8298519.541388347 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.020903798864914 (T) = (0 -9914155.409487586) / Math.Sqrt((160727720358.28433 / (109)) + (22447119097.85811 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.18913959529390717 = (12226710.481788915 - 9914155.409487586) / 12226710.481788915 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 56.47980559395981 (T) = (0 -4686525.524920327) / Math.Sqrt((58545852532.35773 / (109)) + (7644131826.011393 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.33901030123621195 = (7090164.239602027 - 4686525.524920327) / 7090164.239602027 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.84515628647385 (T) = (0 -7934611.729985547) / Math.Sqrt((114651807929.1577 / (109)) + (10669339345.546368 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.22718127239856478 = (10267105.915784243 - 7934611.729985547) / 10267105.915784243 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 22.71579744744644 (T) = (0 -21581645.587980997) / Math.Sqrt((769902483880.4392 / (109)) + (47211127066.39357 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.11396025123179508 = (24357423.713760417 - 21581645.587980997) / 24357423.713760417 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 77.0109356513666 (T) = (0 -5451146.177825616) / Math.Sqrt((110705214932.92195 / (109)) + (3018531184.6761518 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3550720266025175 = (8452333.287869282 - 5451146.177825616) / 8452333.287869282 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 37.24478983256799 (T) = (0 -4169600.9182525375) / Math.Sqrt((77909327917.61078 / (109)) + (18665512576.238 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.35587405927512733 = (6473269.673878126 - 4169600.9182525375) / 6473269.673878126 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 29.65751395620889 (T) = (0 -9336365.90304572) / Math.Sqrt((184504613279.22495 / (108)) + (41893576231.71483 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.2284725832537174 = (12101145.987033654 - 9336365.90304572) / 12101145.987033654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.811755181120585 (T) = (0 -14275113.350669155) / Math.Sqrt((318911495706.4737 / (109)) + (23676339064.82346 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.15592808745640227 = (16912200.4162552 - 14275113.350669155) / 16912200.4162552 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.957086268019225 (T) = (0 -4372750.032311699) / Math.Sqrt((136990044529.22107 / (109)) + (32267948258.16105 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3874581611826833 = (7138696.0942203 - 4372750.032311699) / 7138696.0942203 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 77.81002710190032 (T) = (0 -4256903.188206993) / Math.Sqrt((134097560482.74847 / (109)) + (826660165.3217188 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.40336502545993513 = (7134853.586966743 - 4256903.188206993) / 7134853.586966743 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.663503864911654 (T) = (0 -5109374.791826747) / Math.Sqrt((62148871908.291794 / (109)) + (25108634783.99577 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3123946462295716 = (7430679.188010831 - 5109374.791826747) / 7430679.188010831 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Tests.Perf_Guid

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[EqualsOperator - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_Guid.EqualsOperator.html>)	43.14 ns	9.48 ns	0.22	0.38	False
[EqualsSame - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_Guid.EqualsSame.html>)	49.55 ns	10.67 ns	0.22	0.58	False
[EqualsNotSame - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_Guid.EqualsNotSame.html>)	24.80 ns	9.31 ns	0.38	0.70	False
[NotEqualsOperator - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Tests.Perf_Guid.NotEqualsOperator.html>)	51.05 ns	10.41 ns	0.20	0.45	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Guid*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_Guid*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Guid*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_Guid*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Guid.EqualsOperator ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 136.00380694996653 (T) = (0 -9.529678130909767) / Math.Sqrt((10.739786039771253 / (109)) + (0.033018683099658955 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.8215302481529407 = (53.39660100539771 - 9.529678130909767) / 53.39660100539771 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.EqualsSame ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 101.91864536929934 (T) = (0 -10.415194684127941) / Math.Sqrt((11.573985815332204 / (109)) + (0.05324630436439757 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.7684800470171091 = (44.98616447497989 - 10.415194684127941) / 44.98616447497989 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.EqualsNotSame ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 27.458760142516265 (T) = (0 -10.099089438018153) / Math.Sqrt((11.502267169871073 / (109)) + (0.9986065913405717 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.5864210230355238 = (24.418768845897116 - 10.099089438018153) / 24.418768845897116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.NotEqualsOperator ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 67.50551686652274 (T) = (0 -11.439399563039293) / Math.Sqrt((10.470475711548445 / (102)) + (1.866955601275573 / (6))) is greater than 1.9825972617651593 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (6) - 2, .975) and 0.7914970312643866 = (54.86444453241682 - 11.439399563039293) / 54.86444453241682 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20LongMultiLine).html>)	182.08 μs	70.74 μs	0.39	0.20	False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20LongMultiLine).html>)	191.57 μs	74.61 μs	0.39	0.21	False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20LongSingleLine).html>)	34.28 μs	24.09 μs	0.70	0.18	False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20LongSingleLine).html>)	18.43 μs	6.79 μs	0.37	0.36	False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20LongSingleLine).html>)	17.72 μs	6.30 μs	0.36	0.35	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 78.2615640154501 (T) = (0 -72178.03997091974) / Math.Sqrt((264819224.31515592 / (109)) + (3749912.841000583 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6545798164902726 = (208957.2162157315 - 72178.03997091974) / 208957.2162157315 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 71.66805306777276 (T) = (0 -72508.2194224745) / Math.Sqrt((256681692.6359039 / (109)) + (7554347.570688394 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6526581149941587 = (208751.73007498108 - 72508.2194224745) / 208751.73007498108 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 28.7890157614757 (T) = (0 -24701.287206490106) / Math.Sqrt((4716354.49360174 / (108)) + (695287.5856995157 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.3176577976643073 = (36200.732010912296 - 24701.287206490106) / 36200.732010912296 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 58.28292795402091 (T) = (0 -6532.173278640104) / Math.Sqrt((4121428.707743666 / (109)) + (72517.44061204098 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6658951711712213 = (19551.268688749475 - 6532.173278640104) / 19551.268688749475 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 63.93057731180619 (T) = (0 -6175.700571671249) / Math.Sqrt((4745020.455966387 / (109)) + (4435.013253252941 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6853498356161402 = (19627.19639369759 - 6175.700571671249) / 19627.19639369759 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_FromStream(IndexViewModel).SystemTextJsonReflection.html>)	624.41 μs	517.44 μs	0.83	0.03	False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_FromStream(IndexViewModel).SystemTextJsonSourceGen.html>)	625.26 μs	521.83 μs	0.83	0.02	False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.JsonFromStream(IndexViewModel).JsonNet.html>)	1.01 ms	889.83 μs	0.88	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 30.499263105335647 (T) = (0 -520775.58983796294) / Math.Sqrt((809183698.1225854 / (89)) + (11879307.284877066 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.16308999298301705 = (622259.9628055291 - 520775.58983796294) / 622259.9628055291 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 30.073706043186014 (T) = (0 -524806.1804232805) / Math.Sqrt((745554160.7475497 / (89)) + (18947871.441681597 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.1629166134170749 = (626946.1189112859 - 524806.1804232805) / 626946.1189112859 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 4.0221231999207046 (T) = (0 -925635.4546141266) / Math.Sqrt((3058162298.600821 / (100)) + (964189366.7299982 / (6))) is greater than 1.9830375264834914 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (100) + (6) - 2, .975) and 0.05668975147631337 = (981263.0108310371 - 925635.4546141266) / 981263.0108310371 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToString(Mode%3a%20SourceGen).html>)	280.74 μs	257.25 μs	0.92	0.03	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToString(Mode%3a%20Reflection).html>)	284.68 μs	255.13 μs	0.90	0.03	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeObjectProperty(Mode%3a%20Reflection).html>)	284.68 μs	258.03 μs	0.91	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ArrayList>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ArrayList>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ArrayList>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ArrayList>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.792221111418375 (T) = (0 -255912.3094822614) / Math.Sqrt((55520341.19390052 / (89)) + (775324.6351187795 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.09731430193197775 = (283501.0126226427 - 255912.3094822614) / 283501.0126226427 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.7289332663225 (T) = (0 -254950.59547491043) / Math.Sqrt((53959360.21662141 / (89)) + (402450.2892134999 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.09792892519722854 = (282628.05736305506 - 254950.59547491043) / 282628.05736305506 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 34.411101030677926 (T) = (0 -258363.86256280149) / Math.Sqrt((58656433.23685081 / (89)) + (282467.6915484585 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.1006555980702178 = (287280.22547136917 - 258363.86256280149) / 287280.22547136917 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).SubtractBenchmark.html>)	427.46 ns	10.75 ns	0.03	0.23	False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).AddBenchmark.html>)	427.57 ns	11.19 ns	0.03	0.24	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).OnesComplementBenchmark.html>)	47.02 ns	7.61 ns	0.16	0.42	False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).EqualsAllBenchmark.html>)	51.18 ns	9.01 ns	0.18	0.31	False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).ConditionalSelectBenchmark.html>)	91.01 ns	16.80 ns	0.18	0.22	False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).EqualsStaticBenchmark.html>)	449.57 ns	11.03 ns	0.02	0.18	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).XorBenchmark.html>)	58.50 ns	11.10 ns	0.19	0.31	False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).AndNotBenchmark.html>)	63.87 ns	10.79 ns	0.17	0.42	False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).UnaryNegateOperatorBenchmark.html>)	442.63 ns	7.49 ns	0.02	0.27	False
[GreaterThanBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GreaterThanBenchmark.html>)	442.77 ns	11.23 ns	0.03	0.23	False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).SubtractionOperatorBenchmark.html>)	419.15 ns	11.81 ns	0.03	0.23	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).BitwiseOrBenchmark.html>)	62.97 ns	11.00 ns	0.17	0.39	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).BitwiseAndBenchmark.html>)	62.24 ns	12.17 ns	0.20	0.42	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).OnesComplementOperatorBenchmark.html>)	43.66 ns	7.30 ns	0.17	0.50	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 219.75358367919753 (T) = (0 -10.931699964978108) / Math.Sqrt((335.00353838639893 / (90)) + (0.04832907770230188 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.9748906873036397 = (435.3643644958362 - 10.931699964978108) / 435.3643644958362 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 209.56344994702053 (T) = (0 -11.224350856107316) / Math.Sqrt((410.08376546538767 / (99)) + (0.008305500648191511 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9743625381208089 = (437.8105332344799 - 11.224350856107316) / 437.8105332344799 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.36938636248365 (T) = (0 -7.4877080351182395) / Math.Sqrt((186.7345527601769 / (99)) + (0.02109367918193388 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8559640614796445 = (51.98499841107404 - 7.4877080351182395) / 51.98499841107404 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 87.19390225454221 (T) = (0 -9.491061330168767) / Math.Sqrt((17.353651481743988 / (99)) + (0.5311515967370058 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8251345657009397 = (54.27637181821115 - 9.491061330168767) / 54.27637181821115 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 55.607972133375064 (T) = (0 -15.03231406503499) / Math.Sqrt((190.7483383123939 / (98)) + (0.8021079532628559 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.8421524696505609 = (95.23312801763079 - 15.03231406503499) / 95.23312801763079 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 192.78533835397218 (T) = (0 -11.13779550491992) / Math.Sqrt((546.1152102700354 / (99)) + (0.18877197952201347 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9760590261640845 = (465.21898320657675 - 11.13779550491992) / 465.21898320657675 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.2585874915019 (T) = (0 -11.083318996502257) / Math.Sqrt((163.0407389161254 / (99)) + (0.03686908867120746 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.833835356656549 = (66.70082620159927 - 11.083318996502257) / 66.70082620159927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.19243913177683 (T) = (0 -11.656878204500394) / Math.Sqrt((204.18083437017435 / (99)) + (0.9787984039237904 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8337710810045289 = (70.12545274879629 - 11.656878204500394) / 70.12545274879629 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 229.3601260243782 (T) = (0 -7.615609736172105) / Math.Sqrt((294.1238251470395 / (90)) + (0.2795952074783918 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.9820890210768336 = (425.1922672033272 - 7.615609736172105) / 425.1922672033272 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 208.75055750540167 (T) = (0 -11.057865455088196) / Math.Sqrt((428.2368845301636 / (99)) + (0.03643853887260305 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9751801353520653 = (445.52480893598766 - 11.057865455088196) / 445.52480893598766 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 240.71130978646636 (T) = (0 -11.23709637159642) / Math.Sqrt((281.1248778756245 / (92)) + (0.08976891538931953 / (6))) is greater than 1.9849843115220658 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (92) + (6) - 2, .975) and 0.9740508637131178 = (433.0431752087629 - 11.23709637159642) / 433.0431752087629 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.02541760248029 (T) = (0 -11.045679768421019) / Math.Sqrt((180.48474565384544 / (99)) + (0.03752356920664186 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8372913224611431 = (67.88623652714016 - 11.045679768421019) / 67.88623652714016 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.48606939379433 (T) = (0 -11.262849514127884) / Math.Sqrt((172.71451531478962 / (99)) + (0.27386927035473774 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8346351307036787 = (68.10908243120073 - 11.262849514127884) / 68.10908243120073 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 34.65001282610601 (T) = (0 -7.476985775588452) / Math.Sqrt((157.33146828524661 / (99)) + (0.032203235571099435 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8540557346986722 = (51.23178879383092 - 7.476985775588452) / 51.23178879383092 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Tests.Perf_Encoding.GetString(size%3a%20512%2c%20encName%3a%20%22utf-8%22).html>)	6.04 μs	2.66 μs	0.44	0.22	False
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Tests.Perf_Encoding.GetString(size%3a%20512%2c%20encName%3a%20%22ascii%22).html>)	4.38 μs	1.64 μs	0.37	0.33	False
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Tests.Perf_Encoding.GetString(size%3a%2016%2c%20encName%3a%20%22utf-8%22).html>)	497.18 ns	398.52 ns	0.80	0.07	False
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Tests.Perf_Encoding.GetString(size%3a%2016%2c%20encName%3a%20%22ascii%22).html>)	479.06 ns	382.36 ns	0.80	0.03	False
[GetBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Tests.Perf_Encoding.GetBytes(size%3a%20512%2c%20encName%3a%20%22ascii%22).html>)	3.79 μs	2.37 μs	0.63	0.46	False
[GetBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Tests.Perf_Encoding.GetBytes(size%3a%20512%2c%20encName%3a%20%22utf-8%22).html>)	5.88 μs	4.51 μs	0.77	0.30	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Tests.Perf_Encoding*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Tests.Perf_Encoding*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 8.82039241705438 (T) = (0 -2658.758835979054) / Math.Sqrt((4158301.0430787723 / (109)) + (1078.3850583903456 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.39375301542882385 = (4385.6033986869315 - 2658.758835979054) / 4385.6033986869315 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 8.148268421239402 (T) = (0 -1651.9876999326063) / Math.Sqrt((3480409.477271176 / (109)) + (1749.8837626180066 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.46960588165048944 = (3114.641815926032 - 1651.9876999326063) / 3114.641815926032 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 10.386382439344715 (T) = (0 -386.712961146495) / Math.Sqrt((4577.653868445058 / (109)) + (72.4709291246504 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.16492946852809523 = (463.090178101328 - 386.712961146495) / 463.090178101328 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 9.957502316783081 (T) = (0 -382.9393754334571) / Math.Sqrt((3855.362516496484 / (109)) + (2.937934336439512 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.13473338023480755 = (442.56806709748656 - 382.9393754334571) / 442.56806709748656 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.439882113150475 (T) = (0 -2459.0134125271184) / Math.Sqrt((316235.30734796147 / (109)) + (4185.304149929105 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4492777088657864 = (4465.069695041353 - 2459.0134125271184) / 4465.069695041353 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 27.656887499034735 (T) = (0 -4591.325879675119) / Math.Sqrt((344862.5027076497 / (109)) + (15086.449418252512 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3122014969529086 = (6675.393824404362 - 4591.325879675119) / 6675.393824404362 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>)	378.42 μs	192.61 μs	0.51	0.13	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>)	538.14 μs	391.82 μs	0.73	0.06	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>)	382.83 μs	209.77 μs	0.55	0.09	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>)	388.29 μs	204.47 μs	0.53	0.10	False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>)	340.89 μs	169.58 μs	0.50	0.10	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>)	1.12 ms	980.99 μs	0.88	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadLineTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadLineTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.395958019946185 (T) = (0 -197540.73381358435) / Math.Sqrt((4585860680.927044 / (105)) + (66354691.77651246 / (6))) is greater than 1.9819674897361885 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (6) - 2, .975) and 0.3944930359520778 = (326240.23428729747 - 197540.73381358435) / 326240.23428729747 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 14.169504668665388 (T) = (0 -399603.0950649535) / Math.Sqrt((3398560397.3497257 / (109)) + (116926661.59124735 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.2015330773466993 = (500462.929305919 - 399603.0950649535) / 500462.929305919 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 18.859778298945024 (T) = (0 -209067.16893485383) / Math.Sqrt((4501720930.512367 / (103)) + (598862.024959626 / (6))) is greater than 1.9823833701754254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (103) + (6) - 2, .975) and 0.3738487846027951 = (333892.4588723031 - 209067.16893485383) / 333892.4588723031 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 19.317501428136506 (T) = (0 -203895.74224119133) / Math.Sqrt((4557101404.582018 / (109)) + (165595.99821838082 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.37995946970038147 = (328842.60347087955 - 203895.74224119133) / 328842.60347087955 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 20.018240022937132 (T) = (0 -169467.61636531408) / Math.Sqrt((4227737837.2103977 / (109)) + (40467.31210975655 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4238731001063841 = (294149.8069203277 - 169467.61636531408) / 294149.8069203277 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 7.085713529359129 (T) = (0 -1001858.937182826) / Math.Sqrt((2659809657.2662683 / (107)) + (580357482.084831 / (6))) is greater than 1.9815667570746824 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (107) + (6) - 2, .975) and 0.07234402615716651 = (1079989.7434310752 - 1001858.937182826) / 1079989.7434310752 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToString(Mode%3a%20SourceGen).html>)	7.41 μs	6.82 μs	0.92	0.03	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToString(Mode%3a%20Reflection).html>)	9.11 μs	8.45 μs	0.93	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 15.292165917331502 (T) = (0 -6783.240204413173) / Math.Sqrt((43704.61864199735 / (89)) + (5845.88488829778 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07944326614188964 = (7368.628086597328 - 6783.240204413173) / 7368.628086597328 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 25.225661596713202 (T) = (0 -8451.186175262175) / Math.Sqrt((48997.25412483872 / (89)) + (7.6804873860654705 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.06552228472053712 = (9043.753571731544 - 8451.186175262175) / 9043.753571731544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).OnesComplementOperatorBenchmark.html>)	45.66 ns	7.28 ns	0.16	0.58	False
[InequalityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).InequalityOperatorBenchmark.html>)	51.12 ns	9.17 ns	0.18	0.42	False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).UnaryNegateOperatorBenchmark.html>)	253.22 ns	7.66 ns	0.03	0.25	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).OnesComplementBenchmark.html>)	43.43 ns	7.57 ns	0.17	0.37	False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).ConditionalSelectBenchmark.html>)	90.51 ns	14.67 ns	0.16	0.32	False
[XorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).XorBenchmark.html>)	62.22 ns	12.29 ns	0.20	0.34	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).BitwiseAndBenchmark.html>)	61.77 ns	10.95 ns	0.18	0.39	False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).EqualsStaticBenchmark.html>)	292.64 ns	11.14 ns	0.04	0.23	False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).AndNotBenchmark.html>)	63.71 ns	11.06 ns	0.17	0.38	False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).AddBenchmark.html>)	266.09 ns	11.09 ns	0.04	0.21	False
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).ExclusiveOrOperatorBenchmark.html>)	62.54 ns	11.11 ns	0.18	0.46	False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).MultiplyBenchmark.html>)	273.36 ns	11.05 ns	0.04	0.27	False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).EqualsAllBenchmark.html>)	52.45 ns	9.02 ns	0.17	0.38	False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).MultiplyOperatorBenchmark.html>)	256.08 ns	11.16 ns	0.04	0.13	False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).NegateBenchmark.html>)	267.36 ns	7.72 ns	0.03	0.34	False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).SubtractBenchmark.html>)	269.70 ns	11.31 ns	0.04	0.23	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).BitwiseOrOperatorBenchmark.html>)	64.24 ns	10.97 ns	0.17	0.36	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).BitwiseOrBenchmark.html>)	62.04 ns	11.36 ns	0.18	0.32	False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).EqualityOperatorBenchmark.html>)	52.16 ns	9.17 ns	0.18	0.39	False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).SubtractionOperatorBenchmark.html>)	259.66 ns	11.05 ns	0.04	0.29	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 32.78916510060269 (T) = (0 -7.379072200162741) / Math.Sqrt((178.03240456957985 / (99)) + (0.03389279492974405 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8564904037906917 = (51.41866742764985 - 7.379072200162741) / 51.41866742764985 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.InequalityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 73.65265391548954 (T) = (0 -9.230004592299276) / Math.Sqrt((32.1231157033608 / (99)) + (0.007949848251262224 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.819973241053056 = (51.270181423526395 - 9.230004592299276) / 51.270181423526395 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 82.30838785685448 (T) = (0 -7.594381317098761) / Math.Sqrt((991.6702552928997 / (98)) + (0.006575037143103738 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.971813754807709 = (269.43572176033723 - 7.594381317098761) / 269.43572176033723 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.099631704001816 (T) = (0 -7.469851195071179) / Math.Sqrt((170.04219868487758 / (99)) + (0.06912092237456846 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8535165947186941 = (50.99452173934732 - 7.469851195071179) / 50.99452173934732 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 50.20976383223732 (T) = (0 -15.419711169309751) / Math.Sqrt((208.468156416766 / (99)) + (2.588192512413962 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8383598125903805 = (95.39528143600816 - 15.419711169309751) / 95.39528143600816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.XorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.38316221782242 (T) = (0 -11.277375804796478) / Math.Sqrt((200.23190930844373 / (99)) + (0.260560754741646 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8338745907198795 = (67.8847134442906 - 11.277375804796478) / 67.8847134442906 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.227451103717506 (T) = (0 -10.964427158199149) / Math.Sqrt((175.32694507482626 / (99)) + (0.008988952788181936 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8399705153273669 = (68.51504384100662 - 10.964427158199149) / 68.51504384100662 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 87.10355713096578 (T) = (0 -11.261103958032669) / Math.Sqrt((1143.1272480047644 / (99)) + (0.11960019069151244 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9633783648134033 = (307.49866576559015 - 11.261103958032669) / 307.49866576559015 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 44.02401787765305 (T) = (0 -10.959675083713906) / Math.Sqrt((175.03777519645445 / (99)) + (0.04943185686941285 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8426100952831967 = (69.63391396312237 - 10.959675083713906) / 69.63391396312237 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 84.36568471373859 (T) = (0 -11.093540987339855) / Math.Sqrt((1010.8437430795484 / (99)) + (0.009039486102918333 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9604783137704653 = (280.69503216311676 - 11.093540987339855) / 280.69503216311676 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.82575930432334 (T) = (0 -11.027025592136662) / Math.Sqrt((200.51247970110413 / (99)) + (0.016461570447413212 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8372244524661323 = (67.74374750508724 - 11.027025592136662) / 67.74374750508724 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 85.06683816917513 (T) = (0 -11.50771460658153) / Math.Sqrt((984.7245326872328 / (99)) + (0.8373368780835151 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.959144767247939 = (281.6705188394991 - 11.50771460658153) / 281.6705188394991 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 77.75742776883155 (T) = (0 -9.568415108756128) / Math.Sqrt((26.3546672198783 / (99)) + (0.5132220726285208 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8281686290542867 = (55.684913971727994 - 9.568415108756128) / 55.684913971727994 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 82.61043909567866 (T) = (0 -11.242898214755897) / Math.Sqrt((1042.9846001348747 / (99)) + (0.02946298489977896 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9597666261976263 = (279.4420937697391 - 11.242898214755897) / 279.4420937697391 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 82.46299541201508 (T) = (0 -7.500958360639676) / Math.Sqrt((1027.838128590176 / (99)) + (0.04028423260979796 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9725535594333803 = (273.2943946750725 - 7.500958360639676) / 273.2943946750725 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 81.89018213684353 (T) = (0 -11.12180589629831) / Math.Sqrt((1079.1768663167081 / (98)) + (0.032247241036593655 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.9606913504656611 = (282.9353342852101 - 11.12180589629831) / 282.9353342852101 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.72337643782938 (T) = (0 -11.124959751156398) / Math.Sqrt((184.4618564721842 / (99)) + (0.03704945003277539 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8368105710655117 = (68.17206128972033 - 11.124959751156398) / 68.17206128972033 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.83119251318014 (T) = (0 -11.040924755702704) / Math.Sqrt((164.6405715968744 / (99)) + (0.03104906674983478 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8368005658290141 = (67.65295977763626 - 11.040924755702704) / 67.65295977763626 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 68.68989102299973 (T) = (0 -9.72009078285559) / Math.Sqrt((22.666363466849788 / (90)) + (0.6978315814065452 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.8108863519373862 = (51.39814541379562 - 9.72009078285559) / 51.39814541379562 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 83.1312672973065 (T) = (0 -11.576094455342181) / Math.Sqrt((1013.8201903383354 / (99)) + (0.7672408622055276 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9585472260987946 = (279.2598266869076 - 11.576094455342181) / 279.2598266869076 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromString(Mode%3a%20SourceGen).html>)	49.78 μs	36.05 μs	0.72	0.06	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromString(Mode%3a%20Reflection).html>)	50.33 μs	37.06 μs	0.74	0.06	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>)	37.83 μs	29.20 μs	0.77	0.09	False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromReader(Mode%3a%20Reflection).html>)	50.95 μs	34.68 μs	0.68	0.12	False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>)	37.97 μs	29.31 μs	0.77	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<BinaryData>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<BinaryData>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<BinaryData>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<BinaryData>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.84692801276791 (T) = (0 -36546.98799566226) / Math.Sqrt((8394334.948171651 / (89)) + (296223.5134392405 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.3126098112543901 = (53167.747509395434 - 36546.98799566226) / 53167.747509395434 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 49.587274722167926 (T) = (0 -36392.95393745026) / Math.Sqrt((8067775.858289385 / (89)) + (136203.82743855193 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.31447524458667575 = (53087.73118704927 - 36392.95393745026) / 53087.73118704927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 58.420801653238456 (T) = (0 -29440.239400617287) / Math.Sqrt((1817003.8399309805 / (89)) + (39452.457823321085 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.24586052725371435 = (39038.18917395647 - 29440.239400617287) / 39038.18917395647 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 58.39026656436321 (T) = (0 -35093.9250489069) / Math.Sqrt((6490386.952253018 / (89)) + (185472.46876213085 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.34902184711086587 = (53909.528135706314 - 35093.9250489069) / 53909.528135706314 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 60.39733390870846 (T) = (0 -29532.87241073488) / Math.Sqrt((1807945.7548068622 / (89)) + (32404.586516658506 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.2469577561568962 = (39218.08192328722 - 29532.87241073488) / 39218.08192328722 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>)	153.27 μs	85.48 μs	0.56	0.28	False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>)	145.60 μs	77.68 μs	0.53	0.32	False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>)	139.08 μs	87.31 μs	0.63	0.28	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>)	156.98 μs	97.39 μs	0.62	0.30	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>)	238.86 μs	192.97 μs	0.81	0.15	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>)	152.71 μs	90.48 μs	0.59	0.38	False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>)	148.67 μs	99.69 μs	0.67	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StringReaderReadLineTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StringReaderReadLineTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 62.562931883053395 (T) = (0 -84404.24498865457) / Math.Sqrt((73575322.39387324 / (108)) + (1318048.6249480965 / (6))) is greater than 1.9813718148759247 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (6) - 2, .975) and 0.41299209371788653 = (143787.23708039845 - 84404.24498865457) / 143787.23708039845 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 61.98949826427198 (T) = (0 -78768.61126077421) / Math.Sqrt((68392005.59847745 / (109)) + (3230441.5326293847 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4593842941679928 = (145701.67017169687 - 78768.61126077421) / 145701.67017169687 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 34.49531256291563 (T) = (0 -86714.75011264764) / Math.Sqrt((74335366.62048568 / (109)) + (12361353.840588521 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3971342808639043 = (143837.58664683995 - 86714.75011264764) / 143837.58664683995 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 22.68994419632432 (T) = (0 -100382.06108713253) / Math.Sqrt((59914884.85432253 / (103)) + (26530724.510870524 / (6))) is greater than 1.9823833701754254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (103) + (6) - 2, .975) and 0.335816338781717 = (151135.99889374894 - 100382.06108713253) / 151135.99889374894 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 28.382823372220713 (T) = (0 -193397.2728436303) / Math.Sqrt((283705554.6600324 / (105)) + (335159.7058517498 / (6))) is greater than 1.9819674897361885 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (6) - 2, .975) and 0.19595936278772513 = (240531.71431007094 - 193397.2728436303) / 240531.71431007094 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 25.15501021968091 (T) = (0 -93145.33052352282) / Math.Sqrt((73831579.89557187 / (109)) + (25074711.322958764 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3730983291593927 = (148580.44707174733 - 93145.33052352282) / 148580.44707174733 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 73.65258140486661 (T) = (0 -99334.20642329055) / Math.Sqrt((57032858.56562271 / (109)) + (81073.6280171014 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3520038420542375 = (153294.4373284461 - 99334.20642329055) / 153294.4373284461 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>)	822.53 μs	695.58 μs	0.85	0.04	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToString(Mode%3a%20Reflection).html>)	818.87 μs	693.38 μs	0.85	0.03	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToString(Mode%3a%20SourceGen).html>)	734.66 μs	605.35 μs	0.82	0.03	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeObjectProperty(Mode%3a%20SourceGen).html>)	737.35 μs	608.23 μs	0.82	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 49.15357524962136 (T) = (0 -691222.2467403068) / Math.Sqrt((556970957.5519246 / (89)) + (6588114.755461128 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.16168478371254003 = (824537.3975214657 - 691222.2467403068) / 824537.3975214657 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 49.02068285275881 (T) = (0 -689214.8734624675) / Math.Sqrt((555760155.5772967 / (89)) + (6594183.056823475 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.16159593389344262 = (822055.738187309 - 689214.8734624675) / 822055.738187309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 49.16059866034644 (T) = (0 -603474.531279351) / Math.Sqrt((533555719.30607605 / (89)) + (7711216.035004002 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.18019432630493412 = (736118.9982491129 - 603474.531279351) / 736118.9982491129 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 50.90444826327806 (T) = (0 -605086.0300506452) / Math.Sqrt((542059316.2297032 / (89)) + (5482696.214519198 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.18210424628766247 = (739808.2546635403 - 605086.0300506452) / 739808.2546635403 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.XmlFromStream(IndexViewModel).XmlSerializer.html>)	1.48 ms	1.34 ms	0.90	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 1/3/2023 12:19:11 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 13.402637477231657 (T) = (0 -1352334.2852771506) / Math.Sqrt((2714471957.6261315 / (98)) + (287873987.5079789 / (6))) is greater than 1.9834952585625159 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (98) + (6) - 2, .975) and 0.07937317865670798 = (1468927.750012705 - 1352334.2852771506) / 1468927.750012705 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToString(Mode%3a%20SourceGen).html>)	8.78 μs	7.98 μs	0.91	0.08	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToString(Mode%3a%20Reflection).html>)	12.31 μs	11.02 μs	0.90	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<StructRecord>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<StructRecord>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<StructRecord>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<StructRecord>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.248483205769048 (T) = (0 -8000.202796979258) / Math.Sqrt((64987.84861555646 / (16)) + (12098.488906982102 / (6))) is greater than 2.08596344725343 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (6) - 2, .975) and 0.10663420456127384 = (8955.125479200164 - 8000.202796979258) / 8955.125479200164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 13.920552541678411 (T) = (0 -11076.70915384624) / Math.Sqrt((82056.22128079247 / (16)) + (2607.9490056884347 / (6))) is greater than 2.08596344725343 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (6) - 2, .975) and 0.08570268161700653 = (12114.996873704353 - 11076.70915384624) / 12114.996873704353 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).BitwiseOrOperatorBenchmark.html>)	62.32 ns	11.76 ns	0.19	0.36	False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).OnesComplementOperatorBenchmark.html>)	43.84 ns	7.67 ns	0.17	0.50	False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).OnesComplementBenchmark.html>)	44.78 ns	8.94 ns	0.20	0.49	False
[ExclusiveOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).ExclusiveOrOperatorBenchmark.html>)	79.10 ns	11.60 ns	0.15	0.41	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).BitwiseAndOperatorBenchmark.html>)	60.96 ns	11.15 ns	0.18	0.41	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.69754400401307 (T) = (0 -11.229168445177601) / Math.Sqrt((198.929400110319 / (99)) + (0.13317000874050422 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8344061568717509 = (67.81150937164271 - 11.229168445177601) / 67.81150937164271 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.529620127066078 (T) = (0 -7.691400171647833) / Math.Sqrt((187.3137706016022 / (99)) + (0.36389143940385715 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8513758249201973 = (51.750666858322276 - 7.691400171647833) / 51.750666858322276 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 33.14307093210957 (T) = (0 -7.738657869571671) / Math.Sqrt((165.18222773198482 / (99)) + (0.36378305195760935 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8492098086696179 = (51.320697992989665 - 7.738657869571671) / 51.320697992989665 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.ExclusiveOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.36998528658138 (T) = (0 -11.36374195165921) / Math.Sqrt((166.91606266316944 / (99)) + (0.08120491933765998 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8326495996385785 = (67.90388267442017 - 11.36374195165921) / 67.90388267442017 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.60606138168758 (T) = (0 -11.279238305719531) / Math.Sqrt((177.3230131518445 / (99)) + (0.09738964908587336 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8354794216792039 = (68.55822183974041 - 11.279238305719531) / 68.55822183974041 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>)	8.91 ms	8.14 ms	0.91	0.02	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20Reflection).html>)	8.93 ms	8.18 ms	0.92	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 29.51509143055233 (T) = (0 -8200247.749603175) / Math.Sqrt((29827575122.716137 / (89)) + (1257010597.7133174 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07748976427434247 = (8889058.822369339 - 8200247.749603175) / 8889058.822369339 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 36.67706413073166 (T) = (0 -8179551.399006918) / Math.Sqrt((28525218636.172653 / (89)) + (222609700.0628954 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.07816675534249085 = (8873135.620147748 - 8179551.399006918) / 8873135.620147748 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromString(Mode%3a%20Reflection).html>)	9.41 μs	8.86 μs	0.94	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 20.89826618312018 (T) = (0 -8842.08602091379) / Math.Sqrt((26935.662643534215 / (89)) + (3225.356671226722 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.0641166022492947 = (9447.850065686376 - 8842.08602091379) / 9447.850065686376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.XmlFromStream(MyEventsListerViewModel).XmlSerializer.html>)	14.55 ms	13.69 ms	0.94	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 17.439827298289167 (T) = (0 -13649708.715305606) / Math.Sqrt((136149620927.12424 / (109)) + (2812594191.9211373 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.05029221410156225 = (14372535.339796944 - 13649708.715305606) / 14372535.339796944 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Perf_Convert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[ToBase64String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Perf_Convert.ToBase64String(formattingOptions%3a%20None).html>)	41.51 μs	26.72 μs	0.64	0.23	False
[ToBase64CharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Perf_Convert.ToBase64CharArray(binaryDataSize%3a%201024%2c%20formattingOptions%3a%20None).html>)	41.78 μs	25.24 μs	0.60	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Perf_Convert*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Perf_Convert*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Perf_Convert*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Perf_Convert*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Perf_Convert.ToBase64String(formattingOptions: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 49.91017368185695 (T) = (0 -26281.370315729146) / Math.Sqrt((5785088.92622285 / (105)) + (206400.37078944332 / (6))) is greater than 1.9819674897361885 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (6) - 2, .975) and 0.3622953861954628 = (41212.45126161914 - 26281.370315729146) / 41212.45126161914 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 48.8994431570108 (T) = (0 -25968.211202137154) / Math.Sqrt((6214845.728932061 / (109)) + (216477.35334822504 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3648982854884644 = (40888.27129385034 - 25968.211202137154) / 40888.27129385034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>)	6.08 ms	5.49 ms	0.90	0.02	False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromString(Mode%3a%20Reflection).html>)	6.03 ms	5.42 ms	0.90	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 44.82257610723974 (T) = (0 -5524430.188851874) / Math.Sqrt((12867741522.408602 / (89)) + (807497575.9689386 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.11937908717552348 = (6273335.221091884 - 5524430.188851874) / 6273335.221091884 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 31.55396613232889 (T) = (0 -5487877.364045215) / Math.Sqrt((12312439183.719538 / (89)) + (2158889880.5595164 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.11373549664574571 = (6192143.929126338 - 5487877.364045215) / 6192143.929126338 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[LastIndexOfAnyValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).LastIndexOfAnyValues(Size%3a%20512).html>)	1.56 μs	603.97 ns	0.39	0.27	False
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).IndexOfAnyTwoValues(Size%3a%20512).html>)	1.54 μs	589.53 ns	0.38	0.26	False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).Reverse(Size%3a%2033).html>)	297.54 ns	117.09 ns	0.39	0.29	False
[IndexOfAnyThreeValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).IndexOfAnyThreeValues(Size%3a%20512).html>)	2.18 μs	725.34 ns	0.33	0.38	False
[SequenceCompareTo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).SequenceCompareTo(Size%3a%20512).html>)	785.14 ns	608.37 ns	0.77	0.16	False
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).LastIndexOfValue(Size%3a%20512).html>)	1.02 μs	505.45 ns	0.49	0.15	False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Byte).Reverse(Size%3a%20512).html>)	3.59 μs	839.30 ns	0.23	0.39	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 52.473520540264026 (T) = (0 -613.6747514300221) / Math.Sqrt((42324.87777524218 / (109)) + (156.694395890694 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6351281051650057 = (1681.8909872670342 - 613.6747514300221) / 1681.8909872670342 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 53.784231277737625 (T) = (0 -603.4896678204731) / Math.Sqrt((41005.05341106911 / (109)) + (181.90685384034063 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6424598934067911 = (1687.8936284121353 - 603.4896678204731) / 1687.8936284121353 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.Reverse(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 106.43305737846995 (T) = (0 -116.60656604072368) / Math.Sqrt((149.06877092604392 / (79)) + (4.408359539648576 / (6))) is greater than 1.9889597801747605 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (79) + (6) - 2, .975) and 0.5964312149255122 = (288.9385164395242 - 116.60656604072368) / 288.9385164395242 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 55.624439361072575 (T) = (0 -733.4499886572544) / Math.Sqrt((86690.83241440011 / (109)) + (189.57420795903496 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.6856197449345734 = (2333.002715150206 - 733.4499886572544) / 2333.002715150206 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.SequenceCompareTo(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 8.944501690220125 (T) = (0 -647.1040981393747) / Math.Sqrt((666.137564042056 / (107)) + (1046.462590970808 / (6))) is greater than 1.9815667570746824 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (107) + (6) - 2, .975) and 0.15666874670703113 = (767.3190049729773 - 647.1040981393747) / 767.3190049729773 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.LastIndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 38.473629070969146 (T) = (0 -511.26171077596615) / Math.Sqrt((28437.92695393115 / (109)) + (214.1670446133325 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.5644572543931236 = (1173.8496759108784 - 511.26171077596615) / 1173.8496759108784 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.Reverse(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 99.95312301784743 (T) = (0 -844.9155609241571) / Math.Sqrt((64752.97965985276 / (109)) + (193.24078838380606 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.7475062309876288 = (3346.2828181029663 - 844.9155609241571) / 3346.2828181029663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Char).IndexOfAnyFourValues(Size%3a%20512).html>)	3.11 μs	1.56 μs	0.50	0.22	False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Char).Reverse(Size%3a%20512).html>)	3.57 μs	1.44 μs	0.40	0.18	False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Char).Reverse(Size%3a%2033).html>)	292.28 ns	145.02 ns	0.50	0.13	False
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Memory.Span(Char).LastIndexOfValue(Size%3a%20512).html>)	1.06 μs	881.98 ns	0.83	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Char>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Char>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Char>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Char>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 73.65215480133524 (T) = (0 -1560.4741021384882) / Math.Sqrt((35690.1755466387 / (109)) + (423.17071350888426 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.4849509868001944 = (3029.7584543339867 - 1560.4741021384882) / 3029.7584543339867 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.Reverse(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 103.49578554591804 (T) = (0 -1432.3935280701087) / Math.Sqrt((46891.84848377028 / (109)) + (40.151297209132736 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.601632574878613 = (3595.6592776973225 - 1432.3935280701087) / 3595.6592776973225 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.Reverse(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 99.02763960947065 (T) = (0 -145.5179088825622) / Math.Sqrt((148.7060946663345 / (79)) + (2.279827640634437 / (6))) is greater than 1.9889597801747605 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (79) + (6) - 2, .975) and 0.5058239311361314 = (294.46571384388153 - 145.5179088825622) / 294.46571384388153 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.LastIndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 11/23/2022 1:02:40 AM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 16.315582642471185 (T) = (0 -902.8094132888242) / Math.Sqrt((3442.6372199242983 / (109)) + (536.3085218640308 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.16580906446323948 = (1082.2575202257635 - 902.8094132888242) / 1082.2575202257635 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_ToString(MyEventsListerViewModel).SystemTextJsonSourceGen.html>)	8.29 ms	7.52 ms	0.91	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 34.53501381130639 (T) = (0 -7542913.299303647) / Math.Sqrt((32328445396.7527 / (89)) + (320570811.5505481 / (6))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (89) + (6) - 2, .975) and 0.0854701060170028 = (8247858.652769074 - 7542913.299303647) / 8247858.652769074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20Compiled).html>)	50.74 μs	35.86 μs	0.71	0.07	False
[MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20None).html>)	50.72 μs	36.31 μs	0.72	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 38.85209198655266 (T) = (0 -36066.14275697036) / Math.Sqrt((20446815.906486187 / (109)) + (98749.52560590424 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3273262368185161 = (53616.098517640414 - 36066.14275697036) / 53616.098517640414 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 39.32506658554302 (T) = (0 -36066.67300870724) / Math.Sqrt((20136381.05947617 / (109)) + (104125.43665713802 / (6))) is greater than 1.9811803594144102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (109) + (6) - 2, .975) and 0.3289309864957909 = (53745.10263910587 - 36066.67300870724) / 53745.10263910587 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).AddOperatorBenchmark.html>)	172.58 ns	10.98 ns	0.06	0.29	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).BitwiseAndBenchmark.html>)	62.27 ns	11.07 ns	0.18	0.29	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).BitwiseAndOperatorBenchmark.html>)	63.69 ns	12.18 ns	0.19	0.32	False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).EqualsBenchmark.html>)	59.43 ns	10.51 ns	0.18	0.26	False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).EqualsStaticBenchmark.html>)	185.66 ns	10.93 ns	0.06	0.21	False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).ConditionalSelectBenchmark.html>)	91.76 ns	14.79 ns	0.16	0.24	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).BitwiseOrOperatorBenchmark.html>)	63.88 ns	11.15 ns	0.17	0.36	False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).EqualsAllBenchmark.html>)	55.12 ns	9.21 ns	0.17	0.37	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).BitwiseOrBenchmark.html>)	63.87 ns	11.14 ns	0.17	0.34	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 82.94740752423233 (T) = (0 -11.414454795605552) / Math.Sqrt((397.3605391937192 / (99)) + (0.7252745197888026 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9366137602695752 = (180.07780307129863 - 11.414454795605552) / 180.07780307129863 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.76557917587153 (T) = (0 -11.094813860573515) / Math.Sqrt((175.64218886212643 / (99)) + (0.04422166720067668 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8372616751763604 = (68.17579001502583 - 11.094813860573515) / 68.17579001502583 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 41.77612402348196 (T) = (0 -11.33436552065776) / Math.Sqrt((177.28567353517207 / (99)) + (0.19576862205980716 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8326930666830562 = (67.74594032625122 - 11.33436552065776) / 67.74594032625122 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 89.52965006017986 (T) = (0 -10.511623179193725) / Math.Sqrt((36.7941377766178 / (99)) + (0.013715285423324106 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8389266482175165 = (65.25985250116868 - 10.511623179193725) / 65.25985250116868 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 73.3924442167445 (T) = (0 -10.964510853196062) / Math.Sqrt((609.2042628507218 / (99)) + (0.07434189890528219 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9432502219296338 = (193.20799527358045 - 10.964510853196062) / 193.20799527358045 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 68.91219609606448 (T) = (0 -14.76367271304791) / Math.Sqrt((113.62756057253807 / (90)) + (0.009375222577038404 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.839947861905577 = (92.2428959014472 - 14.76367271304791) / 92.2428959014472 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 37.814425182401095 (T) = (0 -11.69074691430225) / Math.Sqrt((189.3528253420698 / (99)) + (1.7116260659654468 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.827448343809083 = (67.75215707792002 - 11.69074691430225) / 67.75215707792002 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 94.12014421891767 (T) = (0 -9.28961423275552) / Math.Sqrt((23.80588857826779 / (99)) + (0.007912155740945627 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8328296989014111 = (55.56976431643175 - 9.28961423275552) / 55.56976431643175 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 50.87898308449137 (T) = (0 -11.04952682269107) / Math.Sqrt((105.21613927208658 / (92)) + (0.010442980189339705 / (6))) is greater than 1.9849843115220658 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (92) + (6) - 2, .975) and 0.8313095357829118 = (65.50178680207678 - 11.04952682269107) / 65.50178680207678 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	2b74314b3c049f5112e526a27a8cac74f2a62d43
Compare	ddee0f24992fcb8450885e787c21a82931af3b89
Diff	Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).EqualsStaticBenchmark.html>)	185.62 ns	11.31 ns	0.06	0.35	False
[BitwiseOrOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).BitwiseOrOperatorBenchmark.html>)	62.22 ns	10.82 ns	0.17	0.32	False
[BitwiseAndBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).BitwiseAndBenchmark.html>)	61.99 ns	10.88 ns	0.18	0.34	False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).AddOperatorBenchmark.html>)	175.92 ns	11.10 ns	0.06	0.21	False
[BitwiseOrBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).BitwiseOrBenchmark.html>)	62.44 ns	11.43 ns	0.18	0.32	False
[BitwiseAndOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).BitwiseAndOperatorBenchmark.html>)	62.22 ns	11.92 ns	0.19	0.37	False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).EqualsBenchmark.html>)	53.77 ns	11.17 ns	0.21	0.36	False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).EqualsAllBenchmark.html>)	53.96 ns	10.14 ns	0.19	0.43	False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).ConditionalSelectBenchmark.html>)	91.22 ns	15.36 ns	0.17	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

Repro Steps

#### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # Use Interpreter export MONO_ENV_OPTIONS="--interpreter" # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 74.61039326893663 (T) = (0 -11.333075536066934) / Math.Sqrt((574.4673060714365 / (99)) + (0.15407157559755674 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9408063891871482 = (191.4577499233475 - 11.333075536066934) / 191.4577499233475 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseOrOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 42.391879614829435 (T) = (0 -11.163289541423808) / Math.Sqrt((172.64404532879115 / (99)) + (0.07540480844259294 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.834238949906901 = (67.34567339645825 - 11.163289541423808) / 67.34567339645825 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseAndBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.92261768878764 (T) = (0 -11.049325293365385) / Math.Sqrt((191.17989312191065 / (99)) + (0.057311815870895046 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8376475882770934 = (68.05766034583897 - 11.049325293365385) / 68.05766034583897 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 89.76116039209899 (T) = (0 -11.149768749906825) / Math.Sqrt((351.4812681607267 / (99)) + (0.10741049873879298 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.9382988950154277 = (180.70614379912178 - 11.149768749906825) / 180.70614379912178 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseOrBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 43.834733774517 (T) = (0 -11.770414717704703) / Math.Sqrt((130.31184773789786 / (92)) + (0.665477765000259 / (6))) is greater than 1.9849843115220658 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (92) + (6) - 2, .975) and 0.8215087750564969 = (65.94394050144669 - 11.770414717704703) / 65.94394050144669 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseAndOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 40.30480992832204 (T) = (0 -11.886206435225413) / Math.Sqrt((179.50049031628058 / (99)) + (0.7166811568731389 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.824988963011408 = (67.91689621266688 - 11.886206435225413) / 67.91689621266688 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 93.39939604683546 (T) = (0 -10.835827331591574) / Math.Sqrt((31.940238795593046 / (99)) + (0.12565522414474234 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8347739525298455 = (65.58183468952676 - 10.835827331591574) / 65.58183468952676 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 65.71684033062944 (T) = (0 -10.385412369215912) / Math.Sqrt((34.84120449707012 / (99)) + (0.6966821658154225 / (6))) is greater than 1.983264144773014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (6) - 2, .975) and 0.8123507519613404 = (55.34481207767114 - 10.385412369215912) / 55.34481207767114 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/8/2023 2:17:54 AM falls between 2/27/2023 12:59:46 AM and 3/8/2023 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 67.05116030258131 (T) = (0 -14.924324310000314) / Math.Sqrt((115.25916001005082 / (90)) + (0.1835025724456283 / (6))) is greater than 1.98552344186624 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (6) - 2, .975) and 0.837255878694457 = (91.70422986880565 - 14.924324310000314) / 91.70422986880565 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

kotlarmilos commented 1 year ago

Most likely https://github.com/dotnet/runtime/pull/81782.

/cc: @BrzVlad @SamMonoRT