search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 1080 Improvements on 9/26/2023 6:21:18 AM #22616

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
36.15 ms 5.50 ms 0.15 0.02 True
35.60 ms 5.17 ms 0.15 0.01 True
46.65 ms 30.98 ms 0.66 0.01 True
27.66 ms 11.98 ms 0.43 0.06 True
35.34 ms 5.03 ms 0.14 0.01 True
36.39 ms 5.71 ms 0.16 0.01 True
43.78 ms 28.96 ms 0.66 0.01 True
38.76 ms 9.55 ms 0.25 0.01 True
7.65 secs 4.57 secs 0.60 0.01 True
38.63 ms 9.27 ms 0.24 0.06 True
7.48 secs 4.46 secs 0.60 0.01 True
1.85 secs 50.46 ms 0.03 0.15 False
132.71 ms 116.36 ms 0.88 0.01 True
38.79 ms 9.41 ms 0.24 0.09 True
36.02 ms 5.39 ms 0.15 0.02 True
26.80 ms 12.84 ms 0.48 0.01 True
21.22 ms 7.43 ms 0.35 0.01 True
22.26 ms 8.59 ms 0.39 0.01 True
37.66 ms 21.92 ms 0.58 0.01 True
133.10 ms 110.35 ms 0.83 0.03 False
28.46 ms 14.05 ms 0.49 0.01 True
91.88 ms 73.69 ms 0.80 0.02 True
67.26 ms 50.87 ms 0.76 0.02 True
35.70 ms 5.30 ms 0.15 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.502972600694444 < 34.313664317500006. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 333.07813786898305 (T) = (0 -5760099.136204977) / Math.Sqrt((8550722649.199717 / (124)) + (247840411099.468 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.8407065792834316 = (36160307.87896728 - 5760099.136204977) / 36160307.87896728 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.173675265277778 < 33.83809628309524. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 332.872326560392 (T) = (0 -5431551.959269178) / Math.Sqrt((3759144199.5142913 / (125)) + (245832664091.49628 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.8475114623829976 = (35619411.426918685 - 5431551.959269178) / 35619411.426918685 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 30.975929158333336 < 44.06255863811813. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 115.30938050333646 (T) = (0 -30971953.622960925) / Math.Sqrt((2365237898052.776 / (124)) + (53341595528.642006 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.3496443770358737 = (47623104.23611013 - 30971953.622960925) / 47623104.23611013 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.978986921428575 < 26.105224127962956. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 246.35562389714767 (T) = (0 -12054609.536134727) / Math.Sqrt((171447707869.97043 / (124)) + (81695324010.82724 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.567007992432751 = (27840258.7702788 - 12054609.536134727) / 27840258.7702788 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.027129236177885 < 33.76015293433674. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 330.26159793923006 (T) = (0 -5372537.682727537) / Math.Sqrt((3789624643.707002 / (125)) + (249923777644.79016 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.8489645214560619 = (35571362.00395856 - 5372537.682727537) / 35571362.00395856 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.7060905953488374 < 34.50387161784013. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 329.4634293741702 (T) = (0 -5964698.194960607) / Math.Sqrt((8324963198.167638 / (125)) + (253441080489.61688 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.8359808515306857 = (36365864.904343896 - 5964698.194960607) / 36365864.904343896 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.961768481481478 < 41.475040286845235. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 326.8875952416335 (T) = (0 -28982392.2967373) / Math.Sqrt((198686997492.77722 / (125)) + (14590087362.994087 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.33944494980894935 = (43875816.69136402 - 28982392.2967373) / 43875816.69136402 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.551667228205131 < 36.89396099451389. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 684.1115925277898 (T) = (0 -9503703.233113639) / Math.Sqrt((220924670163.0928 / (124)) + (3164583416.3605514 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7576965708836322 = (39222322.47299077 - 9503703.233113639) / 39222322.47299077 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.568480325533334 < 7.280789092979881. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 861.6889188805038 (T) = (0 -4561967739.319285) / Math.Sqrt((79232769437290.73 / (123)) + (369668467421207.1 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.4048173537583346 = (7664819813.0880375 - 4561967739.319285) / 7664819813.0880375 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.269374513392856 < 36.68863434500001. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 352.0717923148989 (T) = (0 -8970280.192694979) / Math.Sqrt((205690747710.959 / (125)) + (169314719763.97794 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.7701658193286774 = (39029356.58435872 - 8970280.192694979) / 39029356.58435872 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.455740447 < 7.0977544346566654. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1044.1319614663764 (T) = (0 -4465496575.6233835) / Math.Sqrt((159917403362251.72 / (124)) + (211379788518016.97 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.40301364737164974 = (7480064755.187708 - 4465496575.6233835) / 7480064755.187708 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 50.459825333333335 < 1.7578836852558333. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 5.840629811160837 (T) = (0 -134352185.3468773) / Math.Sqrt((6.965971716731969E+17 / (125)) + (29277449516270156 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.7786616986511539 = (606999260.9870443 - 134352185.3468773) / 606999260.9870443 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.36244178571428 < 124.84537117051282. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 43.83404475745693 (T) = (0 -116440047.07319903) / Math.Sqrt((11382766483335.14 / (124)) + (1216067761911.75 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.12044890193362273 = (132385767.38654885 - 116440047.07319903) / 132385767.38654885 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.410515890532546 < 36.845514632222226. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 341.6071931338266 (T) = (0 -9089858.442164397) / Math.Sqrt((222607790842.39655 / (125)) + (178907364255.42264 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.7678324437251054 = (39152147.64720048 - 9089858.442164397) / 39152147.64720048 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.392934393055556 < 34.22236731086734. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 338.1208095765247 (T) = (0 -5679942.965051258) / Math.Sqrt((8412218116.088133 / (125)) + (240239856612.07083 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.8425054133155393 = (36064369.47849503 - 5679942.965051258) / 36064369.47849503 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.844477293233084 < 25.396076062454213. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 464.4595612951629 (T) = (0 -12937387.490935264) / Math.Sqrt((68467087769.89269 / (126)) + (10554843297.247946 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.5178752048107909 = (26834105.235881936 - 12937387.490935264) / 26834105.235881936 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.426636078472222 < 20.165958781101196. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 720.3422476324239 (T) = (0 -7393785.300741758) / Math.Sqrt((43855293201.45653 / (124)) + (670167315.9105334 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6538803927204996 = (21361937.160558168 - 7393785.300741758) / 21361937.160558168 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.586109246684352 < 21.194715350606064. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 666.2217302666511 (T) = (0 -8564090.212575037) / Math.Sqrt((44836883819.557976 / (124)) + (2280297504.921087 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6193851557849618 = (22500673.168008476 - 8564090.212575037) / 22500673.168008476 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.91676354545455 < 35.76666301271585. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 117.92325552677141 (T) = (0 -21919814.188117) / Math.Sqrt((2615289422152.264 / (125)) + (3992247798.552294 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.43840045000052974 = (39031039.44463217 - 21919814.188117) / 39031039.44463217 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 110.35183953571428 < 126.46308110684522. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/27/2023 6:04:31 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 13.667488739032143 (T) = (0 -108702388.0059732) / Math.Sqrt((79300867953194.5 / (133)) + (2668877638401.008 / (22))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (22) - 2, .975) and 0.09625525702896763 = (120279967.16043684 - 108702388.0059732) / 120279967.16043684 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.047033525210082 < 27.052693129456653. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 249.5670340630549 (T) = (0 -13784247.421453994) / Math.Sqrt((174353547237.9181 / (125)) + (64059448908.9945 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.5182400778840461 = (28612275.09526268 - 13784247.421453994) / 28612275.09526268 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\\s", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.690695875 < 87.42688677842261. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 51.15289706893483 (T) = (0 -73861939.40630037) / Math.Sqrt((19945592975833.438 / (124)) + (628122712762.1637 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2279658592873104 = (95671856.34836826 - 73861939.40630037) / 95671856.34836826 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\\s", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 50.865700964285715 < 64.01008753169872. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 49.95101808898663 (T) = (0 -50840141.69970847) / Math.Sqrt((20659105418530.32 / (124)) + (22199629606.186337 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2866949951451898 = (71274057.17566322 - 50840141.69970847) / 71274057.17566322 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: NonBacktracking) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.304244480952381 < 33.928456344249085. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 332.0160590181508 (T) = (0 -5543376.575686789) / Math.Sqrt((2640532910.0116324 / (125)) + (247735775036.03583 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.8449542156370962 = (35753158.96826856 - 5543376.575686789) / 35753158.96826856 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.49 Ξs 1.30 Ξs 0.87 0.01 True
936.60 ns 819.66 ns 0.88 0.02 False
1.51 Ξs 1.28 Ξs 0.85 0.02 True
  • [EscapeDataString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=false_MonoAOT=false_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_Uri.EscapeDataString(input%3a%20%22a%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bca%7b%c3%bc.html>)
  • 📝 - Benchmark Source
  • 📈 - ADX Test Multi Config Graph
 108.29 Ξs 39.05 Ξs 0.36 0.02 False
320.68 ns 300.44 ns 0.94 0.01 True
3.04 Ξs 2.81 Ξs 0.93 0.08 False
632.47 ns 528.63 ns 0.84 0.03 False
1.00 Ξs 870.61 ns 0.87 0.01 False
672.28 ns 614.24 ns 0.91 0.01 True
  • [EscapeDataString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=false_MonoAOT=false_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_Uri.EscapeDataString(input%3a%20%22%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b%7b.html>)
  • 📝 - Benchmark Source
  • 📈 - ADX Test Multi Config Graph
 30.05 Ξs 24.26 Ξs 0.81 0.01 False
625.59 ns 556.70 ns 0.89 0.02 True
5.89 Ξs 1.94 Ξs 0.33 0.02 False
6.39 Ξs 2.30 Ξs 0.36 0.02 False
317.50 ns 300.54 ns 0.95 0.01 True
1.14 Ξs 956.54 ns 0.84 0.03 True
2.47 Ξs 456.54 ns 0.18 0.02 False
299.99 ns 273.65 ns 0.91 0.01 True
180.18 ns 168.44 ns 0.93 0.01 False
53.26 Ξs 3.08 Ξs 0.06 0.19 False
5.00 Ξs 1.29 Ξs 0.26 0.02 False
1.09 Ξs 915.22 ns 0.84 0.01 False
588.35 ns 488.19 ns 0.83 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Uri*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_Uri*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Uri*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_Uri*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://a.much.longer.domain.name/path/with?key=value#fragment") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3012651950384113 < 1.4141005346415938. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 53.92884820983771 (T) = (0 -1301.8225517215287) / Math.Sqrt((612.0709517182945 / (125)) + (125.82827359447491 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.11103413818273429 = (1464.423559595733 - 1301.8225517215287) / 1464.423559595733 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://dot.net/path/with?key=value#fragment") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 819.6569053903381 < 881.6771617544462. IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 16.060240664414508 (T) = (0 -819.4212488728461) / Math.Sqrt((899.2117510791716 / (126)) + (166.22720157797744 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.0652326901050041 = (876.6045198616255 - 819.4212488728461) / 876.6045198616255 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://xn--hst-sna.with.xn--nicode-2ya/path/with?key=value#fragment") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.282218459446955 < 1.4361593344373116. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 52.95891925343238 (T) = (0 -1290.4311129767984) / Math.Sqrt((601.1041757488846 / (125)) + (170.745242753383 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1173765789860924 = (1462.0404152593464 - 1290.4311129767984) / 1462.0404152593464 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.EscapeDataString(input: "a{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þa{Þ") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.054338375 < 102.95538457588634. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 5.4860666534637454 (T) = (0 -43049.981508076125) / Math.Sqrt((1027318864.8989478 / (124)) + (62750738.54924497 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2910319436346205 = (60722.03270874808 - 43049.981508076125) / 60722.03270874808 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "https://CONTOSO.com") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 300.43877150059313 < 301.708129618612. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 46.5169052946046 (T) = (0 -300.03170734775085) / Math.Sqrt((39.575119232182246 / (125)) + (0.9352530107110498 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.08377167731595724 = (327.4639081979301 - 300.03170734775085) / 327.4639081979301 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://hÃķst.with.Þnicode/path/with?key=value#fragment") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8131650867301294 < 2.8861623598555646. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 15.062499566525089 (T) = (0 -2806.1099026551656) / Math.Sqrt((4956.222902672802 / (124)) + (4098.353998991807 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.06657972607912216 = (3006.266288676121 - 2806.1099026551656) / 3006.266288676121 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CombineAbsoluteRelative(input: "/new/path") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 528.6296161557171 < 600.7909097774849. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 14.131271939489697 (T) = (0 -536.0406977703168) / Math.Sqrt((866.5634437165061 / (125)) + (193.46867793456784 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.08795334326256349 = (587.7338552918288 - 536.0406977703168) / 587.7338552918288 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://contoso.com/path/with?key=value#fragment") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 870.6076969015422 < 0.9548588956944178. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 17.76931617950986 (T) = (0 -887.1243976359387) / Math.Sqrt((699.5182989468655 / (124)) + (269.65018385932615 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.07116116070689103 = (955.0896884448578 - 887.1243976359387) / 955.0896884448578 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "https://a.much.longer.domain.name") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 614.2379351416802 < 639.591726007749. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 37.29306242322277 (T) = (0 -611.1639633680996) / Math.Sqrt((368.5191075023538 / (125)) + (40.933196321006086 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.11246675458520689 = (688.6096566247151 - 611.1639633680996) / 688.6096566247151 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.256636018826136 < 28.51202822016954. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/27/2023 6:04:31 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 9.639776532642003 (T) = (0 -24100.737879052464) / Math.Sqrt((7279543.54719199 / (132)) + (11284.088850651216 / (22))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (22) - 2, .975) and 0.08622828058050137 = (26375.009607829834 - 24100.737879052464) / 26375.009607829834 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "http://xn--hst-sna.with.xn--nicode-2ya") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 556.7004812326807 < 595.8590760339591. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 59.21050753889857 (T) = (0 -552.9485262777678) / Math.Sqrt((94.94546327540547 / (125)) + (32.39992165871821 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.126816740810131 = (633.2559865964312 - 552.9485262777678) / 633.2559865964312 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path with escapable values?key=Þnicode") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.942923978181968 < 5.586389919068397. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 6.528845163900971 (T) = (0 -2109.1164307236645) / Math.Sqrt((2995364.198587243 / (123)) + (140729.42785164353 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.3453553444384916 = (3221.7729310179916 - 2109.1164307236645) / 3221.7729310179916 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path%20with%20escapable%20values?key=%C3%BCnicode") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3013627268346886 < 6.057663870884719. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 7.220119198539453 (T) = (0 -2487.8505462382245) / Math.Sqrt((3050384.756690573 / (124)) + (145583.04615899894 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.3324702855154738 = (3726.9510139475515 - 2487.8505462382245) / 3726.9510139475515 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "https://contoso.com") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 300.53887206704263 < 301.74177450494574. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 44.44654050601596 (T) = (0 -300.0710944179189) / Math.Sqrt((42.96845884328328 / (124)) + (1.0979038685776725 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.08398148541728737 = (327.5819097986406 - 300.0710944179189) / 327.5819097986406 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "http://hÃķst.with.Þnicode") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 956.5449210331877 < 1.0818394002658378. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 57.68658463566797 (T) = (0 -960.8637462985864) / Math.Sqrt((905.4199280991119 / (125)) + (73.61676134155253 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1575081580030419 = (1140.502137113935 - 960.8637462985864) / 1140.502137113935 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.BuilderToString #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 456.5387162102343 < 2.348557649987717. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 6.623634640962468 (T) = (0 -546.8700992864618) / Math.Sqrt((827193.6650587782 / (125)) + (30070.580506899794 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.5139194916797228 = (1125.0607459580144 - 546.8700992864618) / 1125.0607459580144 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.Ctor(input: "http://dot.net") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 273.6519234944779 < 292.7490999892628. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 57.557141917054935 (T) = (0 -274.1794093679212) / Math.Sqrt((19.76145476312922 / (124)) + (2.8920853263693487 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.09597719932988528 = (303.28815729501883 - 274.1794093679212) / 303.28815729501883 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.UnescapeDataString(input: "%E4%BD%A0%E5%A5%BD") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 168.43772278547726 < 171.40010701594878. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 32.78093413626028 (T) = (0 -170.01586817355593) / Math.Sqrt((15.211302738224491 / (124)) + (1.601998904344341 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.07484953259732562 = (183.7710449964633 - 170.01586817355593) / 183.7710449964633 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.EscapeDataString(input: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.079983336742756 < 50.597315350588154. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 5.816044401952178 (T) = (0 -5266.5045702948) / Math.Sqrt((542343864.1059598 / (124)) + (19768135.760545425 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7124329383037691 = (18314.00487674082 - 5266.5045702948) / 18314.00487674082 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path with escapable values?key=va lue") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2906940226579249 < 4.759944823659485. IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 5.890139393396588 (T) = (0 -1482.500595035044) / Math.Sqrt((2913205.577415918 / (126)) + (132131.23428672628 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.3972875448143628 = (2459.7145492512336 - 1482.500595035044) / 2459.7145492512336 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://CONTOSO.com/path/with?key=value#fragment") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 915.218117765449 < 1.0367396144372396. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 23.067323171816472 (T) = (0 -920.879029872705) / Math.Sqrt((1502.9166868836835 / (124)) + (236.96750188239767 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.10078202746891153 = (1024.0887726928386 - 920.879029872705) / 1024.0887726928386 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 488.1851559240834 < 559.8517099888167. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 11.275030276846762 (T) = (0 -495.55836914951067) / Math.Sqrt((1061.292804978505 / (122)) + (203.9107973615988 / (30))) is greater than 1.9759053308964478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (122) + (30) - 2, .975) and 0.08220191337221071 = (539.9426915023447 - 495.55836914951067) / 539.9426915023447 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
129.82 Ξs 118.07 Ξs 0.91 0.01 False
127.16 Ξs 108.04 Ξs 0.85 0.01 True
112.93 Ξs 99.22 Ξs 0.88 0.01 False
135.87 Ξs 117.62 Ξs 0.87 0.01 True
107.62 Ξs 96.90 Ξs 0.90 0.01 False
110.84 Ξs 95.76 Ξs 0.86 0.01 True
110.38 Ξs 99.15 Ξs 0.90 0.01 False
126.84 Ξs 109.42 Ξs 0.86 0.01 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 118.06749713345866 < 123.29369733727478. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 38.950462230467394 (T) = (0 -118325.9358575542) / Math.Sqrt((10103400.673116194 / (118)) + (267629.40552646684 / (30))) is greater than 1.9763456545936318 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (30) - 2, .975) and 0.09191260020014094 = (130302.36504067013 - 118325.9358575542) / 130302.36504067013 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 108.03705516358642 < 120.7787067748214. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 63.004327708099744 (T) = (0 -108079.30534280813) / Math.Sqrt((10495185.846364316 / (119)) + (428940.68009237037 / (30))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (30) - 2, .975) and 0.15727525348778681 = (128249.82984078277 - 108079.30534280813) / 128249.82984078277 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 99.22427493448635 < 107.30095122733064. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 40.48801517207919 (T) = (0 -99406.46370448038) / Math.Sqrt((9074691.446225239 / (118)) + (308127.0956734299 / (30))) is greater than 1.9763456545936318 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (30) - 2, .975) and 0.10734716924324277 = (111360.72197318565 - 99406.46370448038) / 111360.72197318565 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 117.62064371867167 < 128.30030459071014. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 40.25599547726869 (T) = (0 -117944.3792689615) / Math.Sqrt((10944263.984661626 / (118)) + (558066.591553691 / (30))) is greater than 1.9763456545936318 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (30) - 2, .975) and 0.10224808876147877 = (131377.47499333942 - 117944.3792689615) / 131377.47499333942 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 96.8972069359756 < 102.3751337092472. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 37.318473373061465 (T) = (0 -96555.0573220718) / Math.Sqrt((10005928.038269576 / (117)) + (635013.8806906998 / (30))) is greater than 1.9764595626326966 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (30) - 2, .975) and 0.11209194052773878 = (108744.43169201602 - 96555.0573220718) / 108744.43169201602 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 95.75888984814728 < 104.65432791895533. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 43.648795830563394 (T) = (0 -95582.72204094863) / Math.Sqrt((9775353.626392504 / (119)) + (221653.34350498437 / (30))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (30) - 2, .975) and 0.12021632047719365 = (108643.43618284962 - 95582.72204094863) / 108643.43618284962 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 99.152721278826 < 106.8681644543543. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 40.08377296421429 (T) = (0 -98958.95523506588) / Math.Sqrt((9705338.531804655 / (118)) + (246021.30239248698 / (30))) is greater than 1.9763456545936318 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (30) - 2, .975) and 0.10859086165665638 = (111014.0686003938 - 98958.95523506588) / 111014.0686003938 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.41805815972221 < 120.81261123153142. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 62.02959386497412 (T) = (0 -108829.14412981611) / Math.Sqrt((10355161.34750837 / (120)) + (229439.6275037638 / (30))) is greater than 1.9761224936134032 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (120) + (30) - 2, .975) and 0.14871544198045847 = (127841.08804111292 - 108829.14412981611) / 127841.08804111292 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.31 ms 1.19 ms 0.91 0.01 False
1.23 ms 1.12 ms 0.91 0.01 False
1.30 ms 1.19 ms 0.91 0.01 False
1.24 ms 1.11 ms 0.90 0.01 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1882945830357143 < 1.2402953432942707. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 26.176641257181025 (T) = (0 -1192469.269375042) / Math.Sqrt((2143322378.6412234 / (124)) + (83847099.37837555 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.0895568538021895 = (1309767.9677803365 - 1192469.269375042) / 1309767.9677803365 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1152271845238095 < 1.1733497259672618. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 25.110510634212417 (T) = (0 -1125733.3705323075) / Math.Sqrt((2145118268.498827 / (125)) + (128762750.9431282 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.09364072880378821 = (1242038.7878270017 - 1125733.3705323075) / 1242038.7878270017 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1866564074404764 < 1.2349491867227564. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 26.490832750277043 (T) = (0 -1191223.6927609805) / Math.Sqrt((2175980459.243567 / (125)) + (77009096.94653207 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.09040486163375941 = (1309619.6785974295 - 1191223.6927609805) / 1309619.6785974295 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1129866205357144 < 1.1743898190802426. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 23.8431431554358 (T) = (0 -1128288.385178724) / Math.Sqrt((2221265362.0180445 / (124)) + (151019313.07648876 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.09192466999957943 = (1242505.2723084125 - 1128288.385178724) / 1242505.2723084125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
603.14 ns 235.92 ns 0.39 0.01 True
41.97 ns 26.98 ns 0.64 0.01 True
314.15 ns 125.33 ns 0.40 0.01 True
529.61 ns 124.22 ns 0.23 0.03 True
313.29 ns 124.55 ns 0.40 0.01 True
235.80 ns 38.40 ns 0.16 0.09 True
450.18 ns 64.84 ns 0.14 0.09 True
36.79 ns 22.24 ns 0.60 0.01 True
235.02 ns 40.48 ns 0.17 0.01 True
526.52 ns 96.12 ns 0.18 0.01 True
526.51 ns 107.17 ns 0.20 0.01 True

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Globalization.Tests.StringSearch*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Globalization.Tests.StringSearch*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 235.92036667591415 < 574.8491924728016. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1506.2597198512565 (T) = (0 -236.36149994971) / Math.Sqrt((1.772606043178103 / (124)) + (1.3695779590980708 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6094216878366459 = (605.1577688493237 - 236.36149994971) / 605.1577688493237 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.983151417263453 < 39.85143158832015. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 968.2056794992677 (T) = (0 -27.057669256675126) / Math.Sqrt((0.00960637350477099 / (125)) + (0.004888068253179187 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.356541578560305 = (42.05037707974263 - 27.057669256675126) / 42.05037707974263 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 125.33245028754187 < 298.1421594279507. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1150.2138069401205 (T) = (0 -126.12170551340641) / Math.Sqrt((2.4769141514249506 / (125)) + (0.20202924810358028 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.5977468864879261 = (313.53817105910554 - 126.12170551340641) / 313.53817105910554 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 124.22371327609066 < 503.3153510165159. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 442.3113417710997 (T) = (0 -122.5958093186461) / Math.Sqrt((2.2957591813213747 / (124)) + (24.987104237431335 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7690038523001717 = (530.7266399868936 - 122.5958093186461) / 530.7266399868936 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 124.55450193021615 < 297.59934806188824. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1143.79558199814 (T) = (0 -124.44034046455111) / Math.Sqrt((0.6138494193179627 / (125)) + (0.6694865754168837 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6026474041468145 = (313.1735938388824 - 124.44034046455111) / 313.1735938388824 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 38.39925513468786 < 224.0814664830185. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 774.0616544073011 (T) = (0 -38.89058996947498) / Math.Sqrt((0.3979090776097553 / (126)) + (1.8389385436524355 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.8347968770845889 = (235.41074335131134 - 38.89058996947498) / 235.41074335131134 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 64.84391540707624 < 428.8516523623735. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 857.7228230567696 (T) = (0 -66.1125860163665) / Math.Sqrt((2.2529840511790917 / (124)) + (5.482369511460313 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.8532703775601119 = (450.57422568814536 - 66.1125860163665) / 450.57422568814536 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.235670677404954 < 34.95385562516791. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 477.3792573406919 (T) = (0 -22.19131643325011) / Math.Sqrt((0.030786805486858373 / (125)) + (0.020096483103790544 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.39435555328466276 = (36.64083201555448 - 22.19131643325011) / 36.64083201555448 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.477424984435274 < 223.7691848230297. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1259.8224550493594 (T) = (0 -40.94620221823923) / Math.Sqrt((1.0509664317718508 / (124)) + (0.4648436562018022 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.8264964649811403 = (235.99635715657544 - 40.94620221823923) / 235.99635715657544 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 96.11971693549059 < 499.7525346817574. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1336.1477328219532 (T) = (0 -96.75435982897946) / Math.Sqrt((5.321753826181113 / (126)) + (1.8196414398065808 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.8158270739913074 = (525.3451846902448 - 96.75435982897946) / 525.3451846902448 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 107.1683969955532 < 500.3987888744397. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1156.350356583638 (T) = (0 -108.23991007013313) / Math.Sqrt((9.011663044832021 / (124)) + (1.7682889293701693 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7949046069790594 = (527.7539806029755 - 108.23991007013313) / 527.7539806029755 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
127.51 ns 106.43 ns 0.83 0.03 True
107.73 ns 89.09 ns 0.83 0.05 False
2.30 Ξs 1.78 Ξs 0.77 0.01 False
62.57 ns 44.54 ns 0.71 0.01 True
2.55 Ξs 2.00 Ξs 0.79 0.03 False
2.31 Ξs 1.70 Ξs 0.73 0.03 True
75.17 ns 54.95 ns 0.73 0.01 True
2.56 Ξs 2.00 Ξs 0.78 0.01 False
2.33 Ξs 1.78 Ξs 0.76 0.03 False
1.78 Ξs 1.14 Ξs 0.64 0.01 True
108.17 ns 87.72 ns 0.81 0.05 False
149.98 ns 113.29 ns 0.76 0.06 True

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Tests.Perf_Encoding*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Tests.Perf_Encoding*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 106.43489121532902 < 121.23849034387669. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 43.38288433631126 (T) = (0 -103.89986601666233) / Math.Sqrt((36.67178048332937 / (124)) + (1.0395417694305595 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.19355078230998687 = (128.8362165125193 - 103.89986601666233) / 128.8362165125193 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 89.08846090346155 < 101.96435525875015. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 39.67233789654217 (T) = (0 -86.88636240884125) / Math.Sqrt((35.89908229355592 / (124)) + (0.5202233630312036 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.20187027677652228 = (108.86245666672524 - 86.88636240884125) / 108.86245666672524 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7784 < 2.196655571558906. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 33.63864258012794 (T) = (0 -1779.4203594205487) / Math.Sqrt((36293.157544992166 / (125)) + (117.42093619846682 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.24487474427186823 = (2356.457218087266 - 1779.4203594205487) / 2356.457218087266 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "ascii") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 44.54314778353173 < 59.34524845342494. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 108.23300400378555 (T) = (0 -44.212147352792755) / Math.Sqrt((0.14814899061608705 / (123)) + (0.8072450953835227 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.2910105051807137 = (62.35938286231159 - 44.212147352792755) / 62.35938286231159 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.002605642095651 < 2.441539265644266. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 20.42825040712947 (T) = (0 -2004.0385323625678) / Math.Sqrt((114242.28916519167 / (124)) + (284.1693403921324 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.23721869916274554 = (2627.2780024403687 - 2004.0385323625678) / 2627.2780024403687 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6966314367873727 < 2.21219857097905. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 103.03547202170827 (T) = (0 -1700.6210388910285) / Math.Sqrt((4237.161021495632 / (124)) + (49.61663168516532 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2661276777469064 = (2317.3254901750174 - 1700.6210388910285) / 2317.3254901750174 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 54.953834573275465 < 71.608502888975. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 202.63853046555982 (T) = (0 -54.932435358796134) / Math.Sqrt((0.1791953035431217 / (125)) + (0.26441169889911953 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.27189061898562594 = (75.44530642122255 - 54.932435358796134) / 75.44530642122255 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.003990140469796 < 2.4328292410745025. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 20.045261302292744 (T) = (0 -2006.117577781162) / Math.Sqrt((116859.44265636509 / (124)) + (149.3627128973431 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.23521289308866644 = (2623.10590705832 - 2006.117577781162) / 2623.10590705832 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7777408923041835 < 2.2109278284099267. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 33.91963008848894 (T) = (0 -1787.0898523167698) / Math.Sqrt((36214.21165601864 / (124)) + (50.163045192440514 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.24544918486422493 = (2368.4155082321367 - 1787.0898523167698) / 2368.4155082321367 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1430381864931847 < 1.6908416771514576. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 351.5729565029415 (T) = (0 -1156.915272025562) / Math.Sqrt((263.5183151356554 / (124)) + (25.066894885739814 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.34335456986228063 = (1761.8568848989328 - 1156.915272025562) / 1761.8568848989328 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "ascii") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 87.72302298662187 < 102.40677003817568. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.05679077284066 (T) = (0 -87.96794960380971) / Math.Sqrt((36.11971856860176 / (125)) + (2.296619120084746 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.19415545994662078 = (109.16243174891116 - 87.96794960380971) / 109.16243174891116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 113.29121606175956 < 143.3176221827208. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 63.55852953821257 (T) = (0 -114.41940619751576) / Math.Sqrt((39.984248728883244 / (124)) + (0.5881071630716556 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.24521488995293245 = (151.59202887611374 - 114.41940619751576) / 151.59202887611374 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.30 Ξs 9.62 Ξs 0.78 0.01 True
12.71 Ξs 10.02 Ξs 0.79 0.01 True
3.66 Ξs 3.24 Ξs 0.89 0.01 True
33.80 ms 23.15 ms 0.68 0.01 True
551.60 ns 502.85 ns 0.91 0.01 True
2.57 Ξs 2.30 Ξs 0.89 0.01 True
960.07 ns 827.66 ns 0.86 0.01 True
6.35 Ξs 5.34 Ξs 0.84 0.01 True
6.06 Ξs 5.05 Ξs 0.83 0.01 True
2.88 Ξs 2.61 Ξs 0.91 0.01 True
990.99 ns 843.25 ns 0.85 0.01 True
534.90 ns 489.36 ns 0.91 0.01 True
3.49 ms 2.95 ms 0.85 0.01 True
3.05 ms 2.65 ms 0.87 0.01 True
34.14 ms 23.48 ms 0.69 0.01 True
4.13 Ξs 3.61 Ξs 0.87 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.621540042590311 < 11.673449078553258. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 270.2170588256482 (T) = (0 -9646.713467402218) / Math.Sqrt((8136.1373427527005 / (125)) + (826.6919009601933 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.21236022510165378 = (12247.621025293238 - 9646.713467402218) / 12247.621025293238 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.016928163635878 < 12.08964594620726. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 198.0623566172979 (T) = (0 -10006.098672120454) / Math.Sqrt((10061.156099869384 / (126)) + (2631.9601308000474 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.20397544628501252 = (12570.087977089066 - 10006.098672120454) / 12570.087977089066 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.239102259986635 < 3.4783005774887603. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 65.82471982079757 (T) = (0 -3270.537076723631) / Math.Sqrt((1085.8355112765535 / (124)) + (492.83637598704007 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.09173790331276901 = (3600.8736780412764 - 3270.537076723631) / 3600.8736780412764 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400KB) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.151947701298702 < 32.03972785535715. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 642.3128344304703 (T) = (0 -23158931.137054503) / Math.Sqrt((21124281568.77256 / (124)) + (2889778562.8104467 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.311732986574902 = (33648178.229269184 - 23158931.137054503) / 33648178.229269184 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 502.8471099025922 < 523.1567524539446. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 32.43570360641604 (T) = (0 -509.54954021983673) / Math.Sqrt((22.574963190229067 / (124)) + (15.318594928886679 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.05031324802532584 = (536.5448545642397 - 509.54954021983673) / 536.5448545642397 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.29963137476702 < 2.4445122179414494. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 75.2510139049536 (T) = (0 -2296.6326284635584) / Math.Sqrt((522.6348091986202 / (124)) + (97.39864842238592 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.0821492479649073 = (2502.185266364253 - 2296.6326284635584) / 2502.185266364253 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 827.6579241254638 < 908.9739014581605. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 159.61688550122716 (T) = (0 -826.632535774662) / Math.Sqrt((29.55660533937118 / (124)) + (9.375188047951578 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.12534981808387854 = (945.1007418345631 - 826.632535774662) / 945.1007418345631 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.343361804498409 < 6.064919473235424. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 201.3391110533836 (T) = (0 -5355.523773904277) / Math.Sqrt((1428.0820796810515 / (125)) + (322.44345390287083 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.15040084032549758 = (6303.5888311802 - 5355.523773904277) / 6303.5888311802 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.048664465959488 < 5.765542273626972. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 256.4507677934865 (T) = (0 -5029.603777303447) / Math.Sqrt((784.2218263545568 / (124)) + (238.67880080737993 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.16155297932604962 = (5998.713876114213 - 5029.603777303447) / 5998.713876114213 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.610099622662449 < 2.7364479364963494. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 48.8298778664871 (T) = (0 -2621.926968775789) / Math.Sqrt((1001.3200174985266 / (125)) + (143.50912676665962 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.06245452612994988 = (2796.586450311428 - 2621.926968775789) / 2796.586450311428 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 843.252055291621 < 925.5086587805935. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 126.82822757560422 (T) = (0 -845.7323938484703) / Math.Sqrt((47.98553719896923 / (122)) + (12.53496756319963 / (30))) is greater than 1.9759053308964478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (122) + (30) - 2, .975) and 0.11899135446322127 = (959.9592445919582 - 845.7323938484703) / 959.9592445919582 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 489.3613348077968 < 508.04817622402754. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 52.13332425131009 (T) = (0 -490.99669593306567) / Math.Sqrt((21.582361791242786 / (123)) + (4.070776512323902 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.05591630487500954 = (520.0775084544395 - 490.99669593306567) / 520.0775084544395 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.945679656462585 < 3.3092512213879948. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 61.269820075307635 (T) = (0 -2970569.1293124724) / Math.Sqrt((2529768648.7716217 / (125)) + (348419978.1217261 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.10426872118828699 = (3316361.9487010236 - 2970569.1293124724) / 3316361.9487010236 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.6460081808510636 < 2.911068856581514. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 109.68532803773347 (T) = (0 -2650088.980170073) / Math.Sqrt((1146180770.976238 / (124)) + (75932239.27270187 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.12436090448185193 = (3026462.6074078125 - 2650088.980170073) / 3026462.6074078125 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.477837466666667 < 32.375307904493724. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 605.2677484139717 (T) = (0 -23505200.16568621) / Math.Sqrt((20942014016.05345 / (125)) + (3911734732.4289517 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.3077027188962828 = (33952466.38584553 - 23505200.16568621) / 33952466.38584553 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.6096278044319314 < 3.855079251345591. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 42.134238603147764 (T) = (0 -3619.5917124762855) / Math.Sqrt((3642.330959991718 / (124)) + (740.9572190372018 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.07884854281518164 = (3929.4208180903483 - 3619.5917124762855) / 3929.4208180903483 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
112.63 ns 94.33 ns 0.84 0.01 True
106.81 ns 88.46 ns 0.83 0.01 True
100.96 ns 39.09 ns 0.39 0.01 True
22.18 ns 13.09 ns 0.59 0.01 True
113.03 ns 59.64 ns 0.53 0.05 True
3.37 Ξs 2.84 Ξs 0.84 0.11 False
731.42 ns 673.64 ns 0.92 0.01 False
27.60 ns 20.55 ns 0.74 0.01 True
107.24 ns 88.82 ns 0.83 0.01 True
242.89 ns 100.00 ns 0.41 0.03 True
27.37 ns 19.65 ns 0.72 0.12 True
37.87 ns 29.78 ns 0.79 0.04 True
116.71 ns 52.72 ns 0.45 0.02 True
47.81 ns 15.56 ns 0.33 0.04 True
113.17 ns 94.22 ns 0.83 0.01 True
95.23 ns 41.60 ns 0.44 0.01 True
105.48 ns 88.66 ns 0.84 0.01 True
233.66 ns 170.59 ns 0.73 0.01 True
94.74 ns 67.32 ns 0.71 0.01 True
288.65 ns 161.02 ns 0.56 0.01 True
112.39 ns 94.42 ns 0.84 0.01 True
279.49 ns 194.47 ns 0.70 0.01 True
294.43 ns 175.71 ns 0.60 0.01 True
113.24 ns 94.52 ns 0.83 0.01 True
22.08 ns 12.18 ns 0.55 0.11 True
94.09 ns 67.53 ns 0.72 0.03 True
944.33 ns 840.14 ns 0.89 0.01 True
1.74 Ξs 613.99 ns 0.35 0.01 True
294.74 ns 169.27 ns 0.57 0.01 True
106.45 ns 88.69 ns 0.83 0.01 True
291.62 ns 164.39 ns 0.56 0.01 True
194.21 ns 89.47 ns 0.46 0.02 True
1.85 Ξs 439.47 ns 0.24 0.10 True
23.46 ns 17.02 ns 0.73 0.05 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_String.ToUpper(s: "test") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 94.32987940629843 < 107.34023647039452. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 176.9928238957133 (T) = (0 -94.54377856107321) / Math.Sqrt((0.41297039359466264 / (124)) + (0.2204231225838398 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.1620917626779763 = (112.83309358938583 - 94.54377856107321) / 112.83309358938583 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpperInvariant(s: "test") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.4613465796456 < 101.17092651055349. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 206.59534508284665 (T) = (0 -88.72744499915457) / Math.Sqrt((0.2824973739848799 / (125)) + (0.16079729562232745 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16891955801086014 = (106.7615606339994 - 88.72744499915457) / 106.7615606339994 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.089737212763524 < 95.76185584835474. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1193.3736080963267 (T) = (0 -39.14009332962389) / Math.Sqrt((0.16930171723472334 / (125)) + (0.03939330299826375 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6116111970323623 = (100.77554509954608 - 39.14009332962389) / 100.77554509954608 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpperInvariant(s: "TEST") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.093300200094538 < 21.022272215299466. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 173.8826128263424 (T) = (0 -13.043944988100128) / Math.Sqrt((0.004066089093483493 / (124)) + (0.08118248702586811 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4109479914562033 = (22.143961482019623 - 13.043944988100128) / 22.143961482019623 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I') #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 59.6420749872866 < 107.72798154715453. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 131.7721161915901 (T) = (0 -59.89994730912838) / Math.Sqrt((15.161537449244562 / (126)) + (1.9206130118851512 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.4857386433039956 = (116.47763637923326 - 59.89994730912838) / 116.47763637923326 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8379314874775696 < 3.2143123454325093. IsChangePoint: Marked as a change because one of 9/27/2023 6:04:31 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.5361157247203 (T) = (0 -2827.340464032204) / Math.Sqrt((24412.88444001688 / (133)) + (407.36181759930304 / (22))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (22) - 2, .975) and 0.1479495731404675 = (3318.2783259121757 - 2827.340464032204) / 3318.2783259121757 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Format_MultipleArgs #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 673.6445138777364 < 694.2044283881479. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 31.136482174890133 (T) = (0 -680.2317376631225) / Math.Sqrt((522.2708790678646 / (123)) + (21.997741434968763 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.09267496496058954 = (749.7111965323166 - 680.2317376631225) / 749.7111965323166 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpper(s: "TEST") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.55413534781035 < 26.15042447911813. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 131.5790597660125 (T) = (0 -20.193648195172702) / Math.Sqrt((0.016333809403431637 / (124)) + (0.08738506990626758 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2644512922090108 = (27.453855851121766 - 20.193648195172702) / 27.453855851121766 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLowerInvariant(s: "TEST") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.82421582712279 < 101.53978246363756. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 299.40971987732524 (T) = (0 -88.5737654137184) / Math.Sqrt((0.13163766784915135 / (125)) + (0.07794884877066081 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16961678753896708 = (106.66613207558657 - 88.5737654137184) / 106.66613207558657 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+') #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 99.99795266161595 < 230.7189463709108. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 262.847514863109 (T) = (0 -100.36417802941135) / Math.Sqrt((16.76789047505291 / (124)) + (5.193818742995535 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.5925488874715543 = (246.32201249028267 - 100.36417802941135) / 246.32201249028267 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLower(s: "test") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.647805411719887 < 25.973021416113372. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 44.77237053062523 (T) = (0 -19.883342706491486) / Math.Sqrt((0.14382220908982 / (125)) + (0.8031479908255318 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.27339643917897793 = (27.36477465651889 - 19.883342706491486) / 27.36477465651889 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.PadLeft(n: 18) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 29.784154501918124 < 36.050656901079975. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 145.19876978985258 (T) = (0 -29.94697952780289) / Math.Sqrt((0.287875777610859 / (125)) + (0.024386343294775566 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.2129983416621502 = (38.05199037451968 - 29.94697952780289) / 38.05199037451968 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.IndexOfAny #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 52.72486868155232 < 111.07427221129775. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 158.57594516945517 (T) = (0 -52.36136539782512) / Math.Sqrt((14.969953179299926 / (125)) + (1.9403369348739845 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.5653348780354575 = (120.46369205140977 - 52.36136539782512) / 120.46369205140977 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y') #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.564666174113436 < 45.613098379321634. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 381.2568975404986 (T) = (0 -15.671571178401729) / Math.Sqrt((0.022205211177269965 / (125)) + (0.21109978706239452 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6738785696249318 = (48.054404644239554 - 15.671571178401729) / 48.054404644239554 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLower(s: "TeSt") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 94.22202179662825 < 107.27777832533377. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 119.109019790139 (T) = (0 -94.37432170924617) / Math.Sqrt((0.25771280040775635 / (124)) + (0.6521136595756218 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.16301824199474038 = (112.75552998211596 - 94.37432170924617) / 112.75552998211596 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.CtorCharCount(size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.599782902830206 < 90.36715085843458. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 386.68935231577376 (T) = (0 -41.00094115651385) / Math.Sqrt((0.8625421492132115 / (126)) + (0.37524407074039107 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.5674834574850192 = (94.7962381232938 - 41.00094115651385) / 94.7962381232938 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpperInvariant(s: "TeSt") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.65803926934952 < 101.25744789856954. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 249.9303312065759 (T) = (0 -88.80775052466399) / Math.Sqrt((0.24008157652345086 / (125)) + (0.09666274656534589 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16792907695417383 = (106.73098658414833 - 88.80775052466399) / 106.73098658414833 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 170.59066975698252 < 221.6678965404454. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 125.42984130265991 (T) = (0 -171.0100839833597) / Math.Sqrt((30.51959638058452 / (124)) + (1.1313924661446104 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.28097099472311343 = (237.8347503763168 - 171.0100839833597) / 237.8347503763168 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 67.31859049630319 < 89.94576571133308. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 208.4178884401466 (T) = (0 -67.46491698224045) / Math.Sqrt((0.10888953203514942 / (123)) + (0.4747050191206398 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.2853710886193777 = (94.40552419283189 - 67.46491698224045) / 94.40552419283189 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 161.02286909982695 < 274.87387796047057. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 448.3023997226163 (T) = (0 -159.9141943417657) / Math.Sqrt((3.5931208612299934 / (124)) + (1.6004223340910928 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4457853488673946 = (288.5419828129074 - 159.9141943417657) / 288.5419828129074 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLower(s: "TEST") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 94.41756555106068 < 107.22541828680474. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 176.4312956530637 (T) = (0 -94.45342925974776) / Math.Sqrt((0.3618986928231114 / (125)) + (0.23546801281793533 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16221029198395856 = (112.74121459837656 - 94.45342925974776) / 112.74121459837656 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 194.46899843096236 < 265.05009401133776. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 107.92234244666868 (T) = (0 -199.42919410427976) / Math.Sqrt((65.1975230887607 / (122)) + (2.9790871060985986 / (30))) is greater than 1.9759053308964478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (122) + (30) - 2, .975) and 0.3010863324992572 = (285.34167147914275 - 199.42919410427976) / 285.34167147914275 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 175.71010230203098 < 280.7345943791123. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 256.6612280694021 (T) = (0 -173.4949962042024) / Math.Sqrt((2.989000682758559 / (125)) + (6.0697753988644 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.41302456082905975 = (295.5745413287673 - 173.4949962042024) / 295.5745413287673 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpper(s: "TeSt") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 94.51771951917269 < 107.48314305256399. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 125.39268375684694 (T) = (0 -94.66765318512165) / Math.Sqrt((0.3056840958845675 / (124)) + (0.5593003881471823 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.16138473516522542 = (112.88567851644488 - 94.66765318512165) / 112.88567851644488 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLowerInvariant(s: "test") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.177724845040617 < 20.94877108579518. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 70.98861863500444 (T) = (0 -12.771009440476567) / Math.Sqrt((0.005967926942164105 / (124)) + (0.5097133279032697 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.42048179008997594 = (22.037287564198184 - 12.771009440476567) / 22.037287564198184 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 67.53135322833403 < 89.44103170042125. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 121.33069275531324 (T) = (0 -67.58743838516756) / Math.Sqrt((0.2194512352305856 / (125)) + (1.391023915853975 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.2825395904173694 = (94.20371839678974 - 67.58743838516756) / 94.20371839678974 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Join_Enumerable #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 840.1370591007001 < 896.2919341706486. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 138.82173623971138 (T) = (0 -840.8101457964541) / Math.Sqrt((22.96892955444394 / (124)) + (9.39438813660275 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.10438993155476568 = (938.8127438719932 - 840.8101457964541) / 938.8127438719932 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.PadLeft(n: 2142) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 613.9930225023563 < 1.6200951433349113. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 593.3980197975095 (T) = (0 -616.8064025997315) / Math.Sqrt((181.36079962217198 / (125)) + (61.65261056023747 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6430310640245731 = (1727.899378455137 - 616.8064025997315) / 1727.899378455137 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpper(s: "This is a much longer piece of text that might benefit more from vectorization.") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 169.268217084208 < 279.92582254422524. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 456.67357854502876 (T) = (0 -166.99254148537347) / Math.Sqrt((3.790512645615591 / (124)) + (1.4318063822111267 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4334932750683204 = (294.77592080749383 - 166.99254148537347) / 294.77592080749383 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.68954950004748 < 101.18664333009866. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 180.7328542704836 (T) = (0 -88.73011837060811) / Math.Sqrt((0.28019701133603825 / (125)) + (0.22421714667525894 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16720084734262788 = (106.544438821149 - 88.73011837060811) / 106.544438821149 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 164.391555321077 < 276.5363648315572. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 365.28925720843097 (T) = (0 -163.7415295533448) / Math.Sqrt((2.7134944284315936 / (124)) + (2.954703646099673 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4363032142818473 = (290.47802595634323 - 163.7415295533448) / 290.47802595634323 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 89.46920755702051 < 183.28154303951422. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 262.1725184987832 (T) = (0 -88.22498967227439) / Math.Sqrt((13.77453403500639 / (125)) + (1.7374407655546549 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.5492267639212287 = (195.71922778675648 - 88.22498967227439) / 195.71922778675648 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+') #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 439.4675270726428 < 1.753228187994131. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 306.23764088744394 (T) = (0 -468.88151483337657) / Math.Sqrt((183.99765069370036 / (124)) + (565.574523608078 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7465348427194312 = (1849.8854827385849 - 468.88151483337657) / 1849.8854827385849 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.CtorCharCount(size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.018149677492268 < 22.439438778334875. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 61.573643427625264 (T) = (0 -17.681206363316246) / Math.Sqrt((0.24491161422095686 / (125)) + (0.21948072708371225 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.25115389543552796 = (23.611268397529575 - 17.681206363316246) / 23.611268397529575 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.79 Ξs 1.64 Ξs 0.92 0.01 True
1.14 Ξs 1.02 Ξs 0.89 0.01 True
1.56 Ξs 1.33 Ξs 0.85 0.01 True
1.51 Ξs 1.29 Ξs 0.86 0.01 True
1.87 Ξs 1.77 Ξs 0.95 0.01 False
1.14 Ξs 1.01 Ξs 0.89 0.03 True
1.81 Ξs 1.64 Ξs 0.91 0.01 True
1.88 Ξs 1.74 Ξs 0.92 0.01 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6432264611322753 < 1.6925390445098654. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 29.861419868936323 (T) = (0 -1651.7662942470674) / Math.Sqrt((527.6038881990031 / (125)) + (283.3129273606718 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.06264201583233202 = (1762.1509840914857 - 1651.7662942470674) / 1762.1509840914857 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.017027105965591 < 1.0834469969958687. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 53.52638881829159 (T) = (0 -1006.5463373377835) / Math.Sqrt((355.30793959129994 / (125)) + (64.58531382132098 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.10622865963358202 = (1126.1788019798569 - 1006.5463373377835) / 1126.1788019798569 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3321764585632092 < 1.486365527029054. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 82.41242464462646 (T) = (0 -1323.8890796584776) / Math.Sqrt((395.6710135872268 / (124)) + (113.8260729970437 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.14127950447003973 = (1541.6996409774033 - 1323.8890796584776) / 1541.6996409774033 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2895156603427242 < 1.4669936697799364. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 96.31153795408795 (T) = (0 -1282.872304325925) / Math.Sqrt((485.77845055999376 / (124)) + (44.83087820924496 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.14868328996337635 = (1506.927197835381 - 1282.872304325925) / 1506.927197835381 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7714739095598684 < 1.8065022151252317. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 28.125081394592005 (T) = (0 -1736.5278331087825) / Math.Sqrt((899.5312918573042 / (125)) + (172.68204349560614 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.055078430814644856 = (1837.748115545605 - 1736.5278331087825) / 1837.748115545605 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0124492502159594 < 1.084111551463035. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.38119054578987 (T) = (0 -1013.0611464868314) / Math.Sqrt((345.1797878090909 / (125)) + (208.455517410989 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.09814784153029169 = (1123.3117723040393 - 1013.0611464868314) / 1123.3117723040393 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6425664294707016 < 1.726230097910377. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 40.75159269418122 (T) = (0 -1647.0643171771178) / Math.Sqrt((679.0896204223443 / (124)) + (128.43587198555485 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.07174254208854744 = (1774.3615234538013 - 1647.0643171771178) / 1774.3615234538013 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7358086464837943 < 1.7818470028786024. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 27.36388414757564 (T) = (0 -1728.3135119085757) / Math.Sqrt((830.0430094213388 / (125)) + (253.58860326793223 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.05794595224929545 = (1834.6224572095239 - 1728.3135119085757) / 1834.6224572095239 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
113.50 Ξs 81.58 Ξs 0.72 0.04 True
122.32 Ξs 90.78 Ξs 0.74 0.02 True
162.06 Ξs 128.79 Ξs 0.79 0.03 True
120.62 Ξs 87.83 Ξs 0.73 0.01 True
120.90 Ξs 88.02 Ξs 0.73 0.01 True
338.32 Ξs 294.95 Ξs 0.87 0.01 False
217.15 Ξs 181.05 Ξs 0.83 0.01 False
680.43 Ξs 597.02 Ξs 0.88 0.01 False
117.33 Ξs 84.76 Ξs 0.72 0.01 True
117.86 Ξs 86.33 Ξs 0.73 0.04 True
141.07 Ξs 108.75 Ξs 0.77 0.04 True
115.12 Ξs 82.79 Ξs 0.72 0.04 True
1.10 ms 935.86 Ξs 0.85 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadLineTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadLineTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 81.58168311631943 < 107.83518337452024. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 59.49867937599433 (T) = (0 -81268.94777851111) / Math.Sqrt((38537728.288321 / (124)) + (396168.68610362586 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.294147406405262 = (115135.86337428873 - 81268.94777851111) / 115135.86337428873 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 90.78433758665027 < 116.88642332732371. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 57.3988194322243 (T) = (0 -90533.4748382063) / Math.Sqrt((45173106.29443399 / (124)) + (471151.354295369 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2810049267359618 = (125916.68316614386 - 90533.4748382063) / 125916.68316614386 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 128.78509740925057 < 154.09831007809214. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 43.73647829152867 (T) = (0 -128225.60801984352) / Math.Sqrt((78047304.16033567 / (123)) + (4426808.222364775 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.23174203444712702 = (166904.36516016675 - 128225.60801984352) / 166904.36516016675 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 87.82954441287879 < 114.51748775274548. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 60.74017946097995 (T) = (0 -87723.19341291134) / Math.Sqrt((39638464.05075238 / (125)) + (243800.03185936387 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.28308998673639796 = (122362.90718491642 - 87723.19341291134) / 122362.90718491642 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.01965709796488 < 113.80484407104522. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 59.897350133329816 (T) = (0 -87401.71164246731) / Math.Sqrt((38754428.05208156 / (124)) + (329146.5960956423 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2804663668243501 = (121469.94610484196 - 87401.71164246731) / 121469.94610484196 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 8]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 294.9545693396227 < 318.70998147836275. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.92076094274001 (T) = (0 -293310.0754791928) / Math.Sqrt((323669660.2163593 / (125)) + (3715025.4832344265 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16395560803072742 = (350830.7433153297 - 293310.0754791928) / 350830.7433153297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 181.05174712643677 < 205.21867494936342. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 38.594737000213215 (T) = (0 -184299.17134288605) / Math.Sqrt((92465015.34922244 / (125)) + (2993518.912840045 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1609851318380846 = (219661.38901285775 - 184299.17134288605) / 219661.38901285775 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 597.0210870192308 < 646.6522475223214. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 33.01889904175426 (T) = (0 -597803.6016437488) / Math.Sqrt((1429122505.1326122 / (126)) + (24729596.279270478 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.16153696480542912 = (712975.4998740338 - 597803.6016437488) / 712975.4998740338 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 84.76176632289084 < 111.71445340708398. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 60.139988847350445 (T) = (0 -84446.77154438553) / Math.Sqrt((40266057.53102673 / (124)) + (339952.7858484653 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2922084661098558 = (119310.23119229969 - 84446.77154438553) / 119310.23119229969 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 86.33489524785324 < 111.92283676890221. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 52.51958248116272 (T) = (0 -86482.3290727273) / Math.Sqrt((38946723.37808646 / (125)) + (2772962.3812481943 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.2784994722331001 = (119864.54028023627 - 86482.3290727273) / 119864.54028023627 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 108.74743425287356 < 134.06544771443342. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 50.29075096175901 (T) = (0 -108681.0526617665) / Math.Sqrt((48396796.89365105 / (124)) + (2172170.666386359 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.23940727587776303 = (142889.94519003585 - 108681.0526617665) / 142889.94519003585 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 82.78753994236584 < 109.68488799297747. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 59.236063778322524 (T) = (0 -82866.34242314377) / Math.Sqrt((40484685.69209717 / (124)) + (367301.48601905804 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.29380600006194607 = (117342.17853792677 - 82866.34242314377) / 117342.17853792677 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 935.860087254902 < 1.0440569321726192. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.4076405124631 (T) = (0 -939301.84058424) / Math.Sqrt((4852183545.399902 / (125)) + (28567013.255242866 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.18765719699939895 = (1156287.5144762562 - 939301.84058424) / 1156287.5144762562 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
55.06 ns 44.10 ns 0.80 0.01 True
87.72 ns 73.34 ns 0.84 0.01 True
55.10 ns 44.56 ns 0.81 0.01 True
329.44 ns 97.06 ns 0.29 0.01 True
165.01 ns 72.88 ns 0.44 0.03 False
546.51 ns 396.94 ns 0.73 0.01 True
543.20 ns 404.23 ns 0.74 0.04 False
274.62 ns 90.66 ns 0.33 0.01 True
705.77 ns 492.41 ns 0.70 0.01 True
15.62 ns 8.52 ns 0.55 0.01 True
469.13 ns 305.29 ns 0.65 0.01 True
31.50 ns 18.43 ns 0.58 0.03 True
41.46 ns 22.23 ns 0.54 0.02 True
165.58 ns 72.36 ns 0.44 0.02 False
292.75 ns 62.74 ns 0.21 0.08 True
29.35 ns 19.55 ns 0.67 0.01 True
321.80 ns 125.17 ns 0.39 0.10 True
360.29 ns 129.52 ns 0.36 0.01 True
272.75 ns 93.27 ns 0.34 0.08 True
131.06 ns 21.47 ns 0.16 0.01 True
88.32 ns 73.16 ns 0.83 0.01 True
137.35 ns 48.69 ns 0.35 0.03 False
59.95 ns 45.89 ns 0.77 0.01 True
17.93 ns 10.03 ns 0.56 0.01 True
138.20 ns 48.70 ns 0.35 0.04 False
72.43 ns 51.86 ns 0.72 0.01 True
66.92 ns 12.49 ns 0.19 0.03 True
72.43 ns 51.85 ns 0.72 0.01 True
182.85 ns 63.15 ns 0.35 0.01 True
634.87 ns 436.23 ns 0.69 0.01 True
29.22 ns 20.29 ns 0.69 0.03 True
545.27 ns 398.01 ns 0.73 0.01 True
196.84 ns 79.47 ns 0.40 0.03 True
149.26 ns 35.20 ns 0.24 0.03 False
320.59 ns 123.02 ns 0.38 0.01 True
735.45 ns 530.49 ns 0.72 0.01 True
41.65 ns 22.32 ns 0.54 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Perf_Ascii*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Perf_Ascii*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Perf_Ascii*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Perf_Ascii*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Ascii.ToUpper_Bytes(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 44.10129907844019 < 52.39686182800888. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 402.8273637852431 (T) = (0 -44.17705026393347) / Math.Sqrt((0.03743356386601032 / (125)) + (0.013119033714727587 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.19840151967031533 = (55.11119512821949 - 44.17705026393347) / 55.11119512821949 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.34169890547264 < 83.31393383237416. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 386.33618327235433 (T) = (0 -73.48355510962803) / Math.Sqrt((0.04802633671349321 / (125)) + (0.029114053886881618 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16213209828723815 = (87.70303165858678 - 73.48355510962803) / 87.70303165858678 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Bytes(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 44.56050919670736 < 52.30257028807024. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 247.97501322244798 (T) = (0 -44.5163549791087) / Math.Sqrt((0.03995284601520833 / (125)) + (0.04478681818964753 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.19169279313723528 = (55.07355941052092 - 44.5163549791087) / 55.07355941052092 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 97.058623532354 < 313.01726006868296. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 832.6652953167248 (T) = (0 -94.87748387140927) / Math.Sqrt((7.873714205879236 / (124)) + (0.488235051454502 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7125405911486001 = (330.0552389309876 - 94.87748387140927) / 330.0552389309876 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 72.88460179936042 < 156.64136648264235. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 37.72611717853612 (T) = (0 -72.40049284763695) / Math.Sqrt((816.3404385376784 / (122)) + (0.7598565429471653 / (30))) is greater than 1.9759053308964478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (122) + (30) - 2, .975) and 0.5745485583344182 = (170.173340027241 - 72.40049284763695) / 170.173340027241 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 396.94108291440267 < 517.5107819162. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 60.96919949121299 (T) = (0 -397.25644755591026) / Math.Sqrt((755.8877998554606 / (123)) + (1.6300241695313114 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.27648640999038837 = (549.0656333775746 - 397.25644755591026) / 549.0656333775746 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUtf16(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 404.2314107919929 < 514.2485714240823. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.54368375717324 (T) = (0 -405.6911139483313) / Math.Sqrt((2244.8727517672546 / (124)) + (0.9131988956861711 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.26610674335858886 = (552.7930802974481 - 405.6911139483313) / 552.7930802974481 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 90.66104165308457 < 260.75429676196086. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 667.139676817673 (T) = (0 -91.24068837801448) / Math.Sqrt((5.998491814959296 / (125)) + (0.8384551328506381 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6683139553518177 = (275.0814809673196 - 91.24068837801448) / 275.0814809673196 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 492.4103614935844 < 670.138881201064. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 894.5185788599616 (T) = (0 -492.3895419120614) / Math.Sqrt((3.940766149491652 / (125)) + (0.745189011473708 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.3013387082575233 = (704.7614455411306 - 492.3895419120614) / 704.7614455411306 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Bytes(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.51529724443977 < 14.825758463854939. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 370.7983697131255 (T) = (0 -8.524578730969013) / Math.Sqrt((0.044998148098108974 / (125)) + (0.0003734527033714017 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.45635608943530165 = (15.680445536701205 - 8.524578730969013) / 15.680445536701205 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.FromUtf16(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 305.2864789353016 < 445.4336637476946. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 524.8478672643433 (T) = (0 -304.62037661030445) / Math.Sqrt((4.677093763695412 / (125)) + (1.7949393294490146 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.34950789560226403 = (468.2921968627736 - 304.62037661030445) / 468.2921968627736 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.FromUtf16(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.428039407615007 < 29.92070002016395. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 156.36996449700774 (T) = (0 -18.528107130670087) / Math.Sqrt((0.055645126798952206 / (125)) + (0.19601091020405387 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.4135038913324086 = (31.591185102254904 - 18.528107130670087) / 31.591185102254904 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.227104295014502 < 39.383435424169775. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 214.3406943751246 (T) = (0 -22.476773967842114) / Math.Sqrt((0.0340199258782291 / (124)) + (0.22896794336218582 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4588574260015972 = (41.535770881535655 - 22.476773967842114) / 41.535770881535655 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 72.36145745296963 < 157.12337479987235. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 38.42892165838125 (T) = (0 -73.22454528188517) / Math.Sqrt((802.3435559943424 / (126)) + (0.9848780283157006 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.5703987646271028 = (170.44770650700227 - 73.22454528188517) / 170.44770650700227 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.Equals_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 62.73695058460785 < 277.7290815190442. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 222.42355601525472 (T) = (0 -66.9955748479856) / Math.Sqrt((2.868756923140468 / (124)) + (30.176373195512475 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.771051406940132 = (292.6227846723079 - 66.9955748479856) / 292.6227846723079 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLowerInPlace_Bytes(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.545850115567685 < 27.83031143626891. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 276.23777620365377 (T) = (0 -19.646591881528206) / Math.Sqrt((0.01010315373897955 / (124)) + (0.03448228913714639 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.3303390201405766 = (29.338116558101472 - 19.646591881528206) / 29.338116558101472 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 125.17325581551647 < 305.21667628816425. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 241.77462393924176 (T) = (0 -128.32331697245107) / Math.Sqrt((4.890080972938708 / (124)) + (17.931052866044446 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6006246296818094 = (321.31004190421964 - 128.32331697245107) / 321.31004190421964 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 129.52108173526693 < 342.0945450599302. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 884.2833308753052 (T) = (0 -130.38009620743975) / Math.Sqrt((3.971392230026837 / (124)) + (1.0720086860822227 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6384029354534784 = (360.567352422922 - 130.38009620743975) / 360.567352422922 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLowerInPlace_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 93.26509564915503 < 259.93807276369563. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 306.6495454132984 (T) = (0 -95.86913429443707) / Math.Sqrt((4.736267536951039 / (124)) + (8.997590797273538 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6503407757157391 = (274.17876502665564 - 95.86913429443707) / 274.17876502665564 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.470971643651986 < 124.47356045824516. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1112.6222098686628 (T) = (0 -21.030418868377318) / Math.Sqrt((1.0468247784805917 / (123)) + (0.039480119527093176 / (30))) is greater than 1.975798923817634 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (123) + (30) - 2, .975) and 0.8398594502400559 = (131.3247575326962 - 21.030418868377318) / 131.3247575326962 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.15733497456975 < 83.34245975555926. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 259.7891797760512 (T) = (0 -73.54214994015554) / Math.Sqrt((0.11303813552657585 / (125)) + (0.06062780248067959 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16041021729433194 = (87.59295486321675 - 73.54214994015554) / 87.59295486321675 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpperInPlace_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 48.69342901262173 < 130.6331070373371. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 36.596482270779155 (T) = (0 -48.46589320198345) / Math.Sqrt((844.0942465503458 / (125)) + (0.41242110921193736 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6626405903885065 = (143.66249116275506 - 48.46589320198345) / 143.66249116275506 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUtf16(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 45.88581692936399 < 57.0053307627575. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 251.46086751318003 (T) = (0 -45.88737833134439) / Math.Sqrt((0.32054320314146084 / (124)) + (0.018206960883798747 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.23640911037481474 = (60.094193048673716 - 45.88737833134439) / 60.094193048673716 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.032999151776734 < 16.987227264163778. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1127.0387877473 (T) = (0 -10.04136840236769) / Math.Sqrt((0.00156166150201973 / (125)) + (0.0010795938052822107 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.43867367197472823 = (17.88864676576441 - 10.04136840236769) / 17.88864676576441 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLowerInPlace_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 48.70430674270882 < 131.08501102053367. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 37.177927354514864 (T) = (0 -48.161134542828925) / Math.Sqrt((833.1182769262163 / (124)) + (0.42743603708983646 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6670047842913355 = (144.6301095958232 - 48.161134542828925) / 144.6301095958232 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 51.86395033759773 < 68.57927377031689. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 714.424258468824 (T) = (0 -51.92332216045926) / Math.Sqrt((0.044632579884937476 / (124)) + (0.013855829842921129 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.282864439383979 = (72.40377553702261 - 51.92332216045926) / 72.40377553702261 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.492847169456308 < 63.5668741022809. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 61.750539632587426 (T) = (0 -12.674086010094896) / Math.Sqrt((104.35436866854782 / (126)) + (0.044183545219200815 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.8161063267901394 = (68.92072896728293 - 12.674086010094896) / 68.92072896728293 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 51.852567423661746 < 68.79845192597124. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 408.94447067755107 (T) = (0 -51.89035132369007) / Math.Sqrt((0.09973504157138761 / (125)) + (0.05194518559129993 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.2838501997987369 = (72.45739831122911 - 51.89035132369007) / 72.45739831122911 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 63.15405324115825 < 173.36542349517845. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 49.36758310830142 (T) = (0 -62.684754144442664) / Math.Sqrt((807.8446495395247 / (125)) + (0.8454946341571863 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6673845592192921 = (188.4601448366628 - 62.684754144442664) / 188.4601448366628 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.Equals_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 436.2315292752737 < 603.3192602776702. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 681.5456853930377 (T) = (0 -433.74765361773564) / Math.Sqrt((1.6349154340985788 / (125)) + (2.214568704201074 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.3165638763915946 = (634.6571956536848 - 433.74765361773564) / 634.6571956536848 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpperInPlace_Bytes(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.287840492511755 < 27.822429719287697. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 165.91124572925807 (T) = (0 -20.13315748679996) / Math.Sqrt((0.04462684884006638 / (124)) + (0.0800939873360974 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.312048008234617 = (29.265352419629462 - 20.13315748679996) / 29.265352419629462 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 398.01283793180556 < 517.4006539550809. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 62.38367252852131 (T) = (0 -397.13746829266876) / Math.Sqrt((731.3716283586035 / (124)) + (1.6651188762561926 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.27708333611074565 = (549.3544251090985 - 397.13746829266876) / 549.3544251090985 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 79.46972881870713 < 187.7049433317186. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 45.949408984820955 (T) = (0 -80.33107094498317) / Math.Sqrt((897.646566078625 / (125)) + (0.8028269662067248 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6056290358460834 = (203.69418199264604 - 80.33107094498317) / 203.69418199264604 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.Equals_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.20143751634688 < 141.80746446968092. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 43.386710934816485 (T) = (0 -34.866799517400864) / Math.Sqrt((956.5672866737249 / (124)) + (0.4531847894062788 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7757608293833065 = (155.48933498777936 - 34.866799517400864) / 155.48933498777936 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 123.01870618354725 < 304.5972183566787. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 576.4024889173684 (T) = (0 -123.38399387020907) / Math.Sqrt((6.267982701581662 / (126)) + (2.025534437678556 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.6153458408469309 = (320.7660464191411 - 123.38399387020907) / 320.7660464191411 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 530.488531396465 < 698.4278087132324. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 472.90477047103354 (T) = (0 -529.7141035704096) / Math.Sqrt((3.3012921298457965 / (125)) + (4.863530215191165 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.27934806094897074 = (735.048467735963 - 529.7141035704096) / 735.048467735963 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLowerInPlace_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.3232234283441 < 39.67707055558729. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 368.06639580090825 (T) = (0 -22.439313415581385) / Math.Sqrt((0.014661717348085133 / (124)) + (0.0791049136404881 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4626429122903908 = (41.758662775297225 - 22.439313415581385) / 41.758662775297225 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
331.36 ms 280.42 ms 0.85 0.04 False
336.15 ms 287.25 ms 0.85 0.01 False
338.84 ms 287.81 ms 0.85 0.03 False
333.30 ms 280.59 ms 0.84 0.04 False
332.07 ms 280.73 ms 0.85 0.03 False
340.14 ms 286.26 ms 0.84 0.01 False
338.43 ms 286.78 ms 0.85 0.04 False
331.85 ms 282.51 ms 0.85 0.04 False
332.46 ms 279.87 ms 0.84 0.03 False
337.94 ms 287.89 ms 0.85 0.03 False
337.79 ms 284.96 ms 0.84 0.01 False
331.95 ms 282.06 ms 0.85 0.03 False
336.81 ms 288.23 ms 0.86 0.01 False
331.85 ms 281.88 ms 0.85 0.03 False
339.01 ms 286.79 ms 0.85 0.01 False
334.06 ms 280.43 ms 0.84 0.05 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.IO.Tests.StreamReaderReadToEndTests*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.4199202 < 315.16306485538456. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.583442901068736 (T) = (0 -281030360.7371306) / Math.Sqrt((316982743754415.8 / (126)) + (1931226688560.7769 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.16900911267912158 = (338187054.7860938 - 281030360.7371306) / 338187054.7860938 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 0, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 287.25442439999995 < 320.8313500602564. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.72182300094011 (T) = (0 -286506199.38490844) / Math.Sqrt((331302222687182.6 / (126)) + (1333068901671.0383 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.16539279120941652 = (343282680.0047416 - 286506199.38490844) / 343282680.0047416 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 9, 32]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 287.8063017692308 < 322.0036629957417. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.10211815845348 (T) = (0 -286576239.7553479) / Math.Sqrt((320364898549288.56 / (124)) + (1704257729804.6577 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.16599543489706872 = (343614713.5717167 - 286576239.7553479) / 343614713.5717167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 8]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.59326914285714 < 316.1101889983333. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.34428094335056 (T) = (0 -280908139.1013125) / Math.Sqrt((319742955767988.7 / (125)) + (929781405424.7622 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16836385847921612 = (337777695.1680162 - 280908139.1013125) / 337777695.1680162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 9, 32]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.7323643846154 < 316.3590873492308. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.70694091107197 (T) = (0 -280669471.5585043) / Math.Sqrt((332079396606862.1 / (124)) + (1880056998850.8533 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.169929612080353 = (338127315.0364133 - 280669471.5585043) / 338127315.0364133 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 1, 8]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 286.2619132857143 < 322.50983253976193. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.724377260431986 (T) = (0 -286583788.55735654) / Math.Sqrt((311337676902239.25 / (125)) + (1642931766401.7732 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16588993690156228 = (343580303.4107925 - 286583788.55735654) / 343580303.4107925 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 33, 128]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 286.77644626666665 < 321.5062350982143. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 30.983930555568996 (T) = (0 -287562967.7296886) / Math.Sqrt((322320412385306.9 / (125)) + (21495186810381.75 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16358946725287724 = (343806009.6938417 - 287562967.7296886) / 343806009.6938417 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 33, 128]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 282.5056788 < 315.1536158148809. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.20953847590856 (T) = (0 -280849539.2860623) / Math.Sqrt((339200940205468.6 / (124)) + (1346063482288.224 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.16881845635900697 = (337891934.0001224 - 280849539.2860623) / 337891934.0001224 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 0]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 279.8717362 < 315.9447551410714. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.242203292359484 (T) = (0 -280940861.2462271) / Math.Sqrt((314300329223986.2 / (125)) + (2292376149263.037 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1679936928782689 = (337666744.64058185 - 280940861.2462271) / 337666744.64058185 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [1025, 2048]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 287.8888304285714 < 320.26250603904765. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.42704939987801 (T) = (0 -286665179.90379727) / Math.Sqrt((317750170652286 / (125)) + (1331892848053.7986 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1657978849315697 = (343639958.1416572 - 286665179.90379727) / 343639958.1416572 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 1, 1]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 284.9574644 < 321.42003637582417. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.12758616806547 (T) = (0 -286345831.6484005) / Math.Sqrt((320531196233789.25 / (124)) + (1227675563414.8616 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.16582490093219196 = (343268256.23108673 - 286345831.6484005) / 343268256.23108673 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 282.0571350714286 < 316.4133689696429. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.93592694629357 (T) = (0 -280860984.109017) / Math.Sqrt((310383075888907.44 / (125)) + (1425397720285.239 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1691169670703118 = (338027102.46555763 - 280860984.109017) / 338027102.46555763 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 129, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 288.2267696923077 < 321.34507123379115. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.449271121752695 (T) = (0 -286999030.5994689) / Math.Sqrt((306094640230591.8 / (122)) + (1544322026946.815 / (30))) is greater than 1.9759053308964478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (122) + (30) - 2, .975) and 0.1650277759496884 = (343722847.6981956 - 286999030.5994689) / 343722847.6981956 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 1]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 281.8767843076923 < 315.6672173299999. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.89699433854306 (T) = (0 -280938480.4558913) / Math.Sqrt((325130977324010.94 / (125)) + (1966039347767.3923 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.16863445873656516 = (337924133.86408365 - 280938480.4558913) / 337924133.86408365 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 0, 0]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 286.7882369230769 < 320.96296458999996. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 35.02585306109359 (T) = (0 -286757761.7706166) / Math.Sqrt((318605663229696.44 / (124)) + (1702456752188.2666 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.1652335433472207 = (343518548.79321456 - 286757761.7706166) / 343518548.79321456 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024]) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.4297033333333 < 315.52488862074176. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 31.267070944152906 (T) = (0 -282110002.52118427) / Math.Sqrt((322349345504725.3 / (125)) + (18483696966022.88 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.1653496092943174 = (337997808.01955307 - 282110002.52118427) / 337997808.01955307 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
154.51 ns 33.97 ns 0.22 0.08 True
45.04 ns 28.01 ns 0.62 0.02 False
50.18 ns 23.37 ns 0.47 0.02 True
29.61 ns 15.70 ns 0.53 0.03 True
191.94 ns 64.08 ns 0.33 0.01 True
28.87 ns 15.11 ns 0.52 0.02 True
218.57 ns 87.89 ns 0.40 0.03 True
63.81 ns 47.51 ns 0.74 0.02 False
167.32 ns 49.97 ns 0.30 0.03 True
211.55 ns 81.96 ns 0.39 0.01 True
36.21 ns 23.11 ns 0.64 0.04 True
91.45 ns 68.34 ns 0.75 0.03 True
31.86 ns 13.53 ns 0.42 0.08 False
39.17 ns 23.31 ns 0.60 0.03 False
171.60 ns 39.56 ns 0.23 0.02 True
27.77 ns 20.67 ns 0.74 0.01 True
30.50 ns 14.99 ns 0.49 0.03 True
37.60 ns 18.79 ns 0.50 0.03 False
482.23 ns 43.64 ns 0.09 0.01 True
183.44 ns 58.62 ns 0.32 0.02 True
28.73 ns 15.04 ns 0.52 0.01 True
264.60 ns 40.53 ns 0.15 0.06 True
264.76 ns 41.86 ns 0.16 0.01 True
67.62 ns 50.91 ns 0.75 0.02 False
1.25 Ξs 923.63 ns 0.74 0.01 True
27.80 ns 20.86 ns 0.75 0.01 True
59.17 ns 17.47 ns 0.30 0.02 True
503.25 ns 68.51 ns 0.14 0.08 True
57.15 ns 16.09 ns 0.28 0.02 True
48.17 ns 32.17 ns 0.67 0.03 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Byte>.LastIndexOfValue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 33.97134540377312 < 147.13030460052633. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 237.39900891472473 (T) = (0 -33.09123411169911) / Math.Sqrt((25.792542893362608 / (124)) + (1.7549127788017012 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7873936754700844 = (155.6455772652374 - 33.09123411169911) / 155.6455772652374 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.006908802128073 < 42.81793405615691. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 39.163041315852404 (T) = (0 -28.286328187614107) / Math.Sqrt((33.20060050128396 / (124)) + (0.13673416376961597 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.4194425658893877 = (48.722704293585494 - 28.286328187614107) / 48.722704293585494 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.EndsWith(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.37184785126956 < 47.671999952045624. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 536.3104370542255 (T) = (0 -23.2769969084325) / Math.Sqrt((0.018801307804866874 / (126)) + (0.07055676199437802 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.5353750448351582 = (50.09846468572524 - 23.2769969084325) / 50.09846468572524 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.Reverse(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.695956486620107 < 28.147828159576616. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 184.29115290668807 (T) = (0 -15.875177106020809) / Math.Sqrt((0.12875535074657926 / (124)) + (0.13553529144723103 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.46389770395661073 = (29.61221621915224 - 15.875177106020809) / 29.61221621915224 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 64.07588036464094 < 181.33129319365702. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 245.34252876247828 (T) = (0 -64.74735385358503) / Math.Sqrt((33.87857322176363 / (125)) + (0.2838199752107967 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6674229835336901 = (194.68378946187323 - 64.74735385358503) / 194.68378946187323 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.111642866897123 < 27.449712427856376. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 812.1418753249045 (T) = (0 -15.071873951083576) / Math.Sqrt((0.028972173016295365 / (125)) + (0.001724182268457407 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.4781976628955335 = (28.88425919040324 - 15.071873951083576) / 28.88425919040324 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 87.89416433898627 < 207.60321712921055. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 229.02154240799584 (T) = (0 -88.40840002075447) / Math.Sqrt((35.73452155349674 / (125)) + (1.6842230136920084 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6023823656170438 = (222.34526936399905 - 88.40840002075447) / 222.34526936399905 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 47.50755721205328 < 60.53619713194682. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 38.84543855930344 (T) = (0 -47.36599496865448) / Math.Sqrt((33.179655125076664 / (124)) + (0.07240687733737254 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.29880466295603225 = (67.55035646462727 - 47.36599496865448) / 67.55035646462727 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 49.97464709575156 < 158.70775299654147. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 231.96586532813552 (T) = (0 -49.169769154096805) / Math.Sqrt((24.741474307071105 / (126)) + (2.035122551300518 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.7080192960193448 = (168.40074869247 - 49.169769154096805) / 168.40074869247 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 81.96479546732688 < 201.45288554353755. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 238.0567891425864 (T) = (0 -81.22103046262133) / Math.Sqrt((37.08382153094854 / (125)) + (0.7389449449874812 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.6242550103557792 = (216.15998270403165 - 81.22103046262133) / 216.15998270403165 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.StartsWith(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.114296642415812 < 34.37607120652259. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 133.63914780071238 (T) = (0 -23.31187216669982) / Math.Sqrt((0.0585558836999621 / (125)) + (0.2644251634966379 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.3558034771579233 = (36.18751629371131 - 23.31187216669982) / 36.18751629371131 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.Reverse(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 68.33545193378129 < 87.33189922534555. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 126.1446249875086 (T) = (0 -67.93927560771836) / Math.Sqrt((0.7807672725308067 / (124)) + (0.9254751143960281 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.2635420650970858 = (92.25140009751495 - 67.93927560771836) / 92.25140009751495 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.LastIndexOfValue(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.52771901249122 < 30.397484758619694. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 34.84351686899946 (T) = (0 -14.172259648628291) / Math.Sqrt((25.724688834637924 / (124)) + (2.578939922447751 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.5711411613600906 = (33.046444125004975 - 14.172259648628291) / 33.046444125004975 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.314659879858002 < 37.243095892071366. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 36.52179686850732 (T) = (0 -23.503971967702316) / Math.Sqrt((25.644925969952126 / (125)) + (0.14550568611345632 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.415916072251844 = (40.24074426823932 - 23.503971967702316) / 40.24074426823932 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfValue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.55833117996638 < 162.74685111380347. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 220.7347020039008 (T) = (0 -40.56185875171449) / Math.Sqrt((33.17733067986393 / (124)) + (3.1528093182676105 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7686278108748645 = (175.3100011936914 - 40.56185875171449) / 175.3100011936914 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.StartsWith(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.66631840492913 < 26.36673596845211. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 583.2536939198679 (T) = (0 -20.537445799864106) / Math.Sqrt((0.0055563670812140455 / (124)) + (0.0031984723143495004 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.25896841623964567 = (27.714669995099435 - 20.537445799864106) / 27.714669995099435 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceEqual(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.988615748190153 < 29.07333658944251. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 282.5403091165718 (T) = (0 -15.075773217116367) / Math.Sqrt((0.00716447464901668 / (125)) + (0.08867885646256181 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.5070917152853951 = (30.585351645783913 - 15.075773217116367) / 30.585351645783913 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfValue(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.791293142492123 < 35.708382214058076. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 42.9820594724623 (T) = (0 -18.888655797166905) / Math.Sqrt((33.50239544637937 / (124)) + (0.0856667522223498 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.5431786264302128 = (41.34801235231902 - 18.888655797166905) / 41.34801235231902 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceCompareTo(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 43.643293557453376 < 458.13064666598666. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 3238.717611734459 (T) = (0 -43.640011722833165) / Math.Sqrt((2.0433399510761383 / (126)) + (0.06294798454042848 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.9094508107991104 = (481.948122428957 - 43.640011722833165) / 481.948122428957 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 58.6228121743661 < 174.303555520248. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 227.13441341262157 (T) = (0 -58.70821673510438) / Math.Sqrt((35.202623249111824 / (124)) + (1.1416122157483382 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.6870312298175405 = (187.58490408125297 - 58.70821673510438) / 187.58490408125297 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.044934209673992 < 27.414380945914825. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1129.8267868180571 (T) = (0 -15.057811997780282) / Math.Sqrt((0.003831166326987733 / (124)) + (0.0035542566011106734 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.47836012666480837 = (28.866297933679125 - 15.057811997780282) / 28.866297933679125 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.StartsWith(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.529600717596146 < 251.1722302774622. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1153.3573330521021 (T) = (0 -38.16078797328275) / Math.Sqrt((0.20946369484239766 / (122)) + (1.102033415948591 / (30))) is greater than 1.9759053308964478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (122) + (30) - 2, .975) and 0.8556281137919048 = (264.32284688916826 - 38.16078797328275) / 264.32284688916826 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.EndsWith(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.86196094781606 < 251.53729278923905. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1626.3170446920678 (T) = (0 -42.27024031288779) / Math.Sqrt((0.4337738544685965 / (125)) + (0.45580333883491275 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.8401575535764901 = (264.4494078931378 - 42.27024031288779) / 264.4494078931378 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 50.91078234415115 < 64.2263227416193. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 38.39846572099167 (T) = (0 -51.0599498063834) / Math.Sqrt((33.60826785436725 / (125)) + (0.17735819169798728 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.28274648643929406 = (71.18814873823808 - 51.0599498063834) / 71.18814873823808 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.Reverse(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 923.6296715608077 < 1.1900454034494172. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 425.5803364610732 (T) = (0 -920.298144596551) / Math.Sqrt((18.482649807868206 / (125)) + (14.407199066388365 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.26820065034081597 = (1257.5826215548773 - 920.298144596551) / 1257.5826215548773 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.EndsWith(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.864942411361255 < 26.479028577933395. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 323.0777481588175 (T) = (0 -20.841692883923148) / Math.Sqrt((0.044153668308797356 / (125)) + (0.0038404109548907613 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.25376507848610086 = (27.929131005610486 - 20.841692883923148) / 27.929131005610486 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceEqual(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.46954536612844 < 56.32660451887785. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 1963.3031171065995 (T) = (0 -17.446745196976742) / Math.Sqrt((0.031491271586288734 / (124)) + (0.006051926037927729 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.7060724589717249 = (59.357299883981966 - 17.446745196976742) / 59.357299883981966 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceEqual(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 68.51128801828443 < 478.09706505616003. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 833.027227946325 (T) = (0 -64.89741063377973) / Math.Sqrt((1.0655866535621252 / (124)) + (8.067080530705809 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.8711633801859152 = (503.7186688647115 - 64.89741063377973) / 503.7186688647115 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.SequenceCompareTo(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.085196622673877 < 54.277013439700895. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 581.6347267224091 (T) = (0 -16.080474130981763) / Math.Sqrt((0.01360546820880441 / (125)) + (0.14574626781771516 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.7182449448867892 = (57.07253104836235 - 16.080474130981763) / 57.07253104836235 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.174750169972555 < 45.979061223799086. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 37.6987410172719 (T) = (0 -32.259568283424784) / Math.Sqrt((33.168079413284374 / (124)) + (0.332114055964516 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.3814831144699626 = (52.15632594376139 - 32.259568283424784) / 52.15632594376139 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 736dabeca728ccf8b911d96d1b4c575b4d0db7d2
Compare fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.34 Ξs 1.19 Ξs 0.89 0.01 False
1.82 Ξs 1.63 Ξs 0.89 0.03 False
1.99 Ξs 1.80 Ξs 0.91 0.03 False
1.57 Ξs 1.43 Ξs 0.92 0.02 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps #### Prerequisites (Build files either built locally or downloaded from payload above) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release ` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create mono dotnet mkdir -p $RunDir/artifacts/dotnet-mono $RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages ```
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1867277049133822 < 1.269022781925305. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 21.366218973802393 (T) = (0 -1185.6406165886626) / Math.Sqrt((7208.816565736471 / (126)) + (39.951784836291814 / (30))) is greater than 1.975488058234171 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (126) + (30) - 2, .975) and 0.1211767751532857 = (1349.1229897747228 - 1185.6406165886626) / 1349.1229897747228 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6255949261559337 < 1.7246766095342598. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 20.459785131946855 (T) = (0 -1626.3527555991993) / Math.Sqrt((8468.77784412912 / (125)) + (391.3561043464943 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.10159069851144656 = (1810.2581450398313 - 1626.3527555991993) / 1810.2581450398313 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.8037654235273652 < 1.8801034006623796. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 19.058188964553928 (T) = (0 -1802.9642601960275) / Math.Sqrt((9558.0647260886 / (124)) + (221.11387325540787 / (30))) is greater than 1.975693927815077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (30) - 2, .975) and 0.08853937275056843 = (1978.1043813564816 - 1802.9642601960275) / 1978.1043813564816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4345958143433393 < 1.4993734155821783. IsChangePoint: Marked as a change because one of 8/9/2023 9:46:11 AM, 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/20/2023 11:58:56 PM and 10/2/2023 10:24:51 PM. IsImprovementStdDev: Marked as improvement because 21.85098040464782 (T) = (0 -1420.629166713328) / Math.Sqrt((7152.042734047658 / (125)) + (59.644596200510755 / (30))) is greater than 1.9755903150050484 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (125) + (30) - 2, .975) and 0.10582538952369032 = (1588.7603495659378 - 1420.629166713328) / 1588.7603495659378 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
matouskozak commented 1 year ago

Similar to https://github.com/dotnet/perf-autofiling-issues/issues/22688.