performanceautofiler[bot]
commented
1 year ago Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 0b5f137ab0eea82f59d22da94a0b21f0bc598f16 |
| Compare | a23e797b61acfa9ae5e0575240f464cf62f2d83b |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono |
Improvements in System.Tests.Perf_DateTime
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ToString(format%3a%20%22o%22).html>) | 148.50 ns | 33.02 ns | 0.22 | 0.02 | True | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ToString(format%3a%20%22r%22).html>) | 113.94 ns | 31.02 ns | 0.27 | 0.02 | False | |||||
| [DayOfYear - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.DayOfYear.html>) | 8.83 ns | 0.74 ns | 0.08 | 0.03 | True | |||||
| [Month - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.Month.html>) | 10.07 ns | 1.93 ns | 0.19 | 0.06 | True | |||||
| [Day - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.Day.html>) | 10.48 ns | 1.85 ns | 0.18 | 0.05 | True | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ToString(format%3a%20%22G%22).html>) | 402.03 ns | 135.24 ns | 0.34 | 0.02 | True | |||||
| [op_Subtraction - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.op_Subtraction.html>) | 4.77 ns | 0.00 ns | 0.00 | 0.50 | True | |||||
| [ObjectEquals - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ObjectEquals.html>) | 4.44 ns | 0.96 ns | 0.22 | 0.20 | True | |||||
| [GetUtcNow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.GetUtcNow.html>) | 30.92 ns | 16.28 ns | 0.53 | 0.14 | True | |||||
| [ParseR - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ParseR.html>) | 106.25 ns | 28.22 ns | 0.27 | 0.01 | True | |||||
| [Year - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.Year.html>) | 9.82 ns | 0.75 ns | 0.08 | 0.05 | True | |||||
| [ParseO - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ParseO.html>) | 112.03 ns | 30.64 ns | 0.27 | 0.03 | True | |||||
| [GetNow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.GetNow.html>) | 265.15 ns | 90.94 ns | 0.34 | 0.11 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Tests.Perf_DateTime.ToString(format%3a%20null).html>) | 396.69 ns | 134.17 ns | 0.34 | 0.02 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_DateTime*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_DateTime* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_DateTime*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_DateTime* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_DateTime.ToString(format: "o")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.01768805907825 < 141.18083135778122.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 277.92412334434493 (T) = (0 -33.45759600428513) / Math.Sqrt((43.63790742006916 / (299)) + (0.6225079094465956 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7824393821868129 = (153.78516728158115 - 33.45759600428513) / 153.78516728158115 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.ToString(format: "r")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.020830372983152 < 109.06963313193036.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 82.40925140278601 (T) = (0 -30.808985171209777) / Math.Sqrt((253.4242217989482 / (299)) + (0.11157203229917634 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7120929431627996 = (107.01017720670524 - 30.808985171209777) / 107.01017720670524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.DayOfYear
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7448588789216787 < 8.38914495579089.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 994.7296300891747 (T) = (0 -0.7447215356132768) / Math.Sqrt((0.020191254035610597 / (299)) + (8.339646542861254E-06 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9168151727851296 = (8.952612640399257 - 0.7447215356132768) / 8.952612640399257 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.Month
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.929437633749692 < 9.561037928418491.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 1017.1674655545099 (T) = (0 -1.9251160766534547) / Math.Sqrt((0.018131669062902853 / (299)) + (4.6734734951293655E-05 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.8083884428897654 = (10.046972665359272 - 1.9251160766534547) / 10.046972665359272 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.Day
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8485379400201958 < 9.958310629278268.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 131.56010868082842 (T) = (0 -1.9119194932283214) / Math.Sqrt((0.01931094631308079 / (299)) + (0.06266848748660271 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.817582381641478 = (10.481002385803828 - 1.9119194932283214) / 10.481002385803828 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.ToString(format: "G")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 135.23645508341875 < 378.642352920843.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 150.0346527222312 (T) = (0 -135.68087505455136) / Math.Sqrt((1114.8626501352053 / (299)) + (0.3207582060474809 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.6816666066954173 = (426.22256385377494 - 135.68087505455136) / 426.22256385377494 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.op_Subtraction
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.530472755195902.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 158.7643654166515 (T) = (0 -0.005363001526954105) / Math.Sqrt((0.26254508347976924 / (299)) + (6.176232054410798E-05 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9988639981745978 = (4.720944462439482 - 0.005363001526954105) / 4.720944462439482 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.ObjectEquals
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9598138265985159 < 4.214110278625466.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 201.1082700571317 (T) = (0 -0.9480219430517844) / Math.Sqrt((0.10345586240330984 / (299)) + (0.00018711884606777273 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.8006566275945375 = (4.755723411378417 - 0.9480219430517844) / 4.755723411378417 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.GetUtcNow
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.2775162065786 < 29.425708652897193.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 226.79284875881325 (T) = (0 -16.106975856200155) / Math.Sqrt((0.6078187588139664 / (299)) + (0.03262970029052325 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4773701557749788 = (30.81908933096673 - 16.106975856200155) / 30.81908933096673 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.ParseR
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.21564206272901 < 100.40495481906184.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 649.9077186969952 (T) = (0 -28.40849767594432) / Math.Sqrt((1.8756071447332703 / (299)) + (0.12490838890405555 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7343450359268126 = (106.93757511761697 - 28.40849767594432) / 106.93757511761697 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.Year
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7457864786866409 < 9.322231213642514.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 862.1886553800709 (T) = (0 -0.7448741633173964) / Math.Sqrt((0.032114846360619266 / (299)) + (2.3102432596308636E-06 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9231041624023003 = (9.686794325778681 - 0.7448741633173964) / 9.686794325778681 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.ParseO
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.636131525151207 < 106.78585748027788.
IsChangePoint: Marked as a change because one of 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 659.3899955400541 (T) = (0 -30.808436419154432) / Math.Sqrt((3.6200384918413335 / (299)) + (0.03956606674140471 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7221391189754393 = (110.8771997898877 - 30.808436419154432) / 110.8771997898877 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.GetNow
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 90.9427689634855 < 252.86313548883314.
IsChangePoint: Marked as a change because one of 3/11/2023 9:11:51 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 80.8509330568247 (T) = (0 -90.41945943758809) / Math.Sqrt((1230.7223504798271 / (299)) + (0.641450622675619 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.6458328908130907 = (255.3016841263761 - 90.41945943758809) / 255.3016841263761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_DateTime.ToString(format: null)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 134.17369751050356 < 377.0763467150828.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 145.20643345729272 (T) = (0 -133.30302384858427) / Math.Sqrt((1195.5609743637297 / (299)) + (1.869961279460141 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.6886564938821936 = (428.15418092627567 - 133.30302384858427) / 428.15418092627567 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
### JIT Disasms
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 0b5f137ab0eea82f59d22da94a0b21f0bc598f16 |
| Compare | a23e797b61acfa9ae5e0575240f464cf62f2d83b |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono |
Improvements in System.Collections.CopyTo<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).List(Size%3a%202048).html>) | 2.27 μs | 372.86 ns | 0.16 | 0.05 | False | |||||
| [Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).Array(Size%3a%202048).html>) | 2.34 μs | 353.43 ns | 0.15 | 0.07 | False | |||||
| [ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).ImmutableArray(Size%3a%202048).html>) | 2.35 μs | 378.81 ns | 0.16 | 0.06 | False | |||||
| [ReadOnlyMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).ReadOnlyMemory(Size%3a%202048).html>) | 2.29 μs | 410.51 ns | 0.18 | 0.06 | False | |||||
| [ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).ReadOnlySpan(Size%3a%202048).html>) | 2.27 μs | 375.30 ns | 0.17 | 0.04 | False | |||||
| [Memory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).Memory(Size%3a%202048).html>) | 2.30 μs | 402.31 ns | 0.18 | 0.06 | False | |||||
| [Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Collections.CopyTo(String).Span(Size%3a%202048).html>) | 2.27 μs | 379.06 ns | 0.17 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.CopyTo<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.CopyTo<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.CopyTo<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.CopyTo<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Collections.CopyTo<String>.List(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 372.85889056611853 < 2.1554874527136345.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 69.38834143635434 (T) = (0 -376.24709244088535) / Math.Sqrt((91261.33885269647 / (299)) + (8.9925204020499 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7633209103522095 = (1589.6930016115507 - 376.24709244088535) / 1589.6930016115507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Collections.CopyTo<String>.Array(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 353.4305779842343 < 2.2262825533738937.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 74.98803586339766 (T) = (0 -348.5547307370608) / Math.Sqrt((89929.49521349164 / (299)) + (21.63535111334968 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7890309685012339 = (1652.1606430140878 - 348.5547307370608) / 1652.1606430140878 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Collections.CopyTo<String>.ImmutableArray(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 378.80851748643187 < 2.235649740546789.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 73.70988793839436 (T) = (0 -378.4873804887349) / Math.Sqrt((91476.15044657438 / (299)) + (19.50136025267303 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7734279398512122 = (1670.4945006907985 - 378.4873804887349) / 1670.4945006907985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Collections.CopyTo<String>.ReadOnlyMemory(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 410.51053947462077 < 2.178232315483784.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 68.43076117599317 (T) = (0 -402.92338431774414) / Math.Sqrt((91274.09985185166 / (299)) + (96.62517387005552 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7499051711402949 = (1611.082428832508 - 402.92338431774414) / 1611.082428832508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Collections.CopyTo<String>.ReadOnlySpan(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 375.3046982017959 < 2.1558959076227677.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 68.34697558880549 (T) = (0 -383.28532767904306) / Math.Sqrt((91367.70992912255 / (299)) + (72.33718651741033 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7585504356928647 = (1587.4343313847778 - 383.28532767904306) / 1587.4343313847778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Collections.CopyTo<String>.Memory(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 402.30993733743213 < 2.1825538758653216.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 68.7354786548026 (T) = (0 -396.08856770012613) / Math.Sqrt((91666.17182062882 / (299)) + (121.88660134308056 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7548113793864292 = (1615.4443330564714 - 396.08856770012613) / 1615.4443330564714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Collections.CopyTo<String>.Span(Size: 2048)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 379.06094439733937 < 2.1572002828955625.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/11/2023 4:41:38 AM, 5/15/2023 6:46:12 PM falls between 5/7/2023 5:11:33 AM and 5/15/2023 6:46:12 PM.
IsImprovementStdDev: Marked as improvement because 68.300113728259 (T) = (0 -381.78743561622224) / Math.Sqrt((91908.70986441364 / (299)) + (90.34341738733524 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7600225900612017 = (1590.9307284947768 - 381.78743561622224) / 1590.9307284947768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
Improvements in System.Text.Tests.Perf_Encoding
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare
Repro Steps
#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Tests.Perf_Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Tests.Perf_Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```