performanceautofiler[bot]
commented
1 year ago Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 7a57f6dd800e77508a4ba405e623059935c82f43 |
| Compare | 887c043eb94be364188e2b23a87efa214ea57f1e |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono |
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20Compiled).html>) | 572.29 ns | 1.29 μs | 2.25 | 0.03 | True | |||||
| [MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 769.82 ns | 1.74 μs | 2.27 | 0.01 | True | |||||
| [Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 416.41 ns | 489.89 ns | 1.18 | 0.03 | False | |||||
| [MatchesWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options%3a%20None).html>) | 170.41 μs | 230.69 μs | 1.35 | 0.02 | False | |||||
| [OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 430.64 ns | 1.08 μs | 2.50 | 0.02 | False | |||||
| [IP_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options%3a%20Compiled).html>) | 484.90 ns | 750.18 ns | 1.55 | 0.01 | True | |||||
| [SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 21.79 μs | 39.76 μs | 1.82 | 0.01 | False | |||||
| [IP_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 489.11 ns | 761.45 ns | 1.56 | 0.02 | True | |||||
| [ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20Compiled).html>) | 17.18 μs | 30.13 μs | 1.75 | 0.02 | False | |||||
| [MatchesWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options%3a%20Compiled).html>) | 14.99 μs | 27.94 μs | 1.86 | 0.02 | True | |||||
| [MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20None).html>) | 4.83 μs | 6.53 μs | 1.35 | 0.01 | False | |||||
| [MatchesWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 20.33 μs | 38.53 μs | 1.89 | 0.01 | True | |||||
| [SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20None).html>) | 171.48 μs | 232.70 μs | 1.36 | 0.01 | False | |||||
| [SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20Compiled).html>) | 16.39 μs | 29.56 μs | 1.80 | 0.02 | False | |||||
| [Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 557.39 ns | 741.12 ns | 1.33 | 0.03 | False | |||||
| [ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20None).html>) | 172.42 μs | 231.49 μs | 1.34 | 0.02 | False | |||||
| [Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 551.30 ns | 1.01 μs | 1.83 | 0.01 | True | |||||
| [IP_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 497.12 ns | 761.00 ns | 1.53 | 0.02 | True | |||||
| [OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20Compiled).html>) | 341.43 ns | 778.28 ns | 2.28 | 0.02 | True | |||||
| [Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20Compiled).html>) | 551.08 ns | 755.83 ns | 1.37 | 0.02 | False | |||||
| [Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20Compiled).html>) | 625.83 ns | 800.77 ns | 1.28 | 0.04 | False | |||||
| [ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 22.52 μs | 40.74 μs | 1.81 | 0.01 | False | |||||
| [IP_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options%3a%20Compiled).html>) | 484.26 ns | 757.26 ns | 1.56 | 0.01 | True | |||||
| [MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20IgnoreCase%2c%20Compiled).html>) | 10.13 μs | 20.67 μs | 2.04 | 0.01 | True | |||||
| [Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20Compiled).html>) | 405.04 ns | 493.14 ns | 1.22 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.RegularExpressions.Tests.Perf_Regex_Common* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.RegularExpressions.Tests.Perf_Regex_Common* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2860160637332179 > 607.3851504068558.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -173.09519787035202 (T) = (0 -1292.399964157456) / Math.Sqrt((25.945980539833812 / (20)) + (62.65837485702075 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -1.2301578475337775 = (579.5105335645447 - 1292.399964157456) / 579.5105335645447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7446261578847875 > 804.4021069560849.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -198.66096816720977 (T) = (0 -1740.2209896308277) / Math.Sqrt((41.5437687672291 / (20)) + (87.95711664434917 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -1.2729024150090862 = (765.638233361581 - 1740.2209896308277) / 765.638233361581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 489.8902451638566 > 431.8049071607993.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -24.404658280640415 (T) = (0 -497.61027834537197) / Math.Sqrt((11.525195906940164 / (20)) + (49.43988774729283 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.21417487596906684 = (409.83410890314735 - 497.61027834537197) / 409.83410890314735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 230.68647549019607 > 179.51691233865492.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -103.64675132648438 (T) = (0 -231269.79991515836) / Math.Sqrt((129295.42519646467 / (20)) + (1334378.2327768167 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.3538119089614285 = (170828.60505531827 - 231269.79991515836) / 170828.60505531827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0782254376292215 > 452.20639815281334.
IsChangePoint: Marked as a change because one of 3/28/2023 2:42:21 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -143.02957795749495 (T) = (0 -1071.907792252583) / Math.Sqrt((28.840270665640173 / (20)) + (55.902441371388484 / (3))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (3) - 2, .025) and -1.486775026674144 = (431.0433315257195 - 1071.907792252583) / 431.0433315257195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 750.1780866559045 > 510.4803956111694.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -48.636811371129895 (T) = (0 -759.7560744194675) / Math.Sqrt((39.79007650873354 / (20)) + (116.85361866693587 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.5566444436132058 = (488.0729684525513 - 759.7560744194675) / 488.0729684525513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.75759409987277 > 22.894813728030584.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -410.8153755648958 (T) = (0 -39857.92095836013) / Math.Sqrt((10693.32954216556 / (20)) + (5608.600750771533 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.8301724544164663 = (21778.23235300985 - 39857.92095836013) / 21778.23235300985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 761.44786205672 > 511.3735336568395.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -131.08879946781084 (T) = (0 -759.2021540632351) / Math.Sqrt((28.941570151254364 / (20)) + (11.480145625177942 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.5594804061637569 = (486.8301974571351 - 759.2021540632351) / 486.8301974571351 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.12709061692507 > 17.86990053159545.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -301.1483612156073 (T) = (0 -30022.13233205956) / Math.Sqrt((9066.295265202183 / (20)) + (5571.496613863267 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.7574685356437117 = (17082.600184966206 - 30022.13233205956) / 17082.600184966206 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.935903877520758 > 15.661154961621708.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -97.2747707239246 (T) = (0 -27917.600843324723) / Math.Sqrt((40080.295797277264 / (20)) + (62848.96395461995 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.8648916251707028 = (14970.092881815202 - 27917.600843324723) / 14970.092881815202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.530762391684139 > 5.088140735880446.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -202.23585088070027 (T) = (0 -6549.797592526391) / Math.Sqrt((121.96437058723096 / (20)) + (260.22123269547257 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.3521781376491394 = (4843.886622744613 - 6549.797592526391) / 4843.886622744613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.528310503685496 > 21.458525112080913.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -302.33464963516025 (T) = (0 -38416.74770887406) / Math.Sqrt((7669.0393259555885 / (20)) + (12656.184082225676 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.8823037074402957 = (20409.431037627914 - 38416.74770887406) / 20409.431037627914 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 232.69811911764705 > 180.30689013534703.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -67.82813206706032 (T) = (0 -232851.30607876906) / Math.Sqrt((101759.63593747115 / (20)) + (3216436.1970930006 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.35507672438867566 = (171836.25243346774 - 232851.30607876906) / 171836.25243346774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.561065049826105 > 17.226320223399508.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -157.9826240968781 (T) = (0 -29518.354804542716) / Math.Sqrt((14409.440932866553 / (20)) + (24578.602858431623 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.7967727772705989 = (16428.54075816019 - 29518.354804542716) / 16428.54075816019 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 741.1150646265045 > 580.2572279417716.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -100.44542916505759 (T) = (0 -746.3470870517443) / Math.Sqrt((20.92947664856624 / (20)) + (13.631456971932238 / (5))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (5) - 2, .025) and -0.353927861712436 = (551.2458293810212 - 746.3470870517443) / 551.2458293810212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.48741438065608 > 181.4097145963929.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -168.04887326801753 (T) = (0 -232424.88167283632) / Math.Sqrt((107784.08288962235 / (20)) + (482201.7094653125 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.34514682903127736 = (172787.7408299135 - 232424.88167283632) / 172787.7408299135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0110508872662547 > 575.3918523816478.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -65.49240443083679 (T) = (0 -990.7638652362518) / Math.Sqrt((31.316863964640913 / (20)) + (219.33393125151287 / (5))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (5) - 2, .025) and -0.8036167239644483 = (549.3206245385098 - 990.7638652362518) / 549.3206245385098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 761.000345091397 > 515.5374668464614.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -69.94405712312853 (T) = (0 -766.1028537928282) / Math.Sqrt((25.881813833098917 / (20)) + (57.51945698595643 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.5660578911944107 = (489.1919118063587 - 766.1028537928282) / 489.1919118063587 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 778.2788156288154 > 357.9045717079674.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -61.17477503419956 (T) = (0 -780.852697338659) / Math.Sqrt((9.122866758079258 / (19)) + (205.035140150628 / (4))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (4) - 2, .025) and -1.2910739587695181 = (340.823871857038 - 780.852697338659) / 340.823871857038 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 755.8287323620373 > 573.8744367482457.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -54.5663417213032 (T) = (0 -749.2163602795785) / Math.Sqrt((26.44200871151724 / (20)) + (60.60151709804062 / (5))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (5) - 2, .025) and -0.36430774040234704 = (549.154958293081 - 749.2163602795785) / 549.154958293081 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 800.7738887177908 > 661.6232957360315.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -44.677994244731885 (T) = (0 -792.8199730683116) / Math.Sqrt((69.1256957007973 / (20)) + (46.57791435065859 / (5))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (5) - 2, .025) and -0.25218175421462874 = (633.1508747830063 - 792.8199730683116) / 633.1508747830063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.74478432720999 > 23.61974135299371.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -214.0677328952694 (T) = (0 -40563.936780098615) / Math.Sqrt((3113.0477755386273 / (20)) + (27930.688363213198 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.8046340357276547 = (22477.65251958281 - 40563.936780098615) / 22477.65251958281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 757.2619727926987 > 502.5721014028.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -79.06316819739882 (T) = (0 -757.2642812846641) / Math.Sqrt((70.08526001879336 / (20)) + (32.893545439376716 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.5565322790308207 = (486.50727741809305 - 757.2642812846641) / 486.50727741809305 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.671891642011836 > 10.623149085099655.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -146.8946086684902 (T) = (0 -20740.28268084026) / Math.Sqrt((5374.037683260885 / (20)) + (19720.68428463209 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -1.0436373775244747 = (10148.709799956609 - 20740.28268084026) / 10148.709799956609 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```
#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 493.1404498868985 > 425.01726105386234.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsRegressionStdDev: Marked as regression because -43.566428461641046 (T) = (0 -497.4694330250045) / Math.Sqrt((12.554183677770743 / (20)) + (18.899350266012732 / (5))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (5) - 2, .025) and -0.22527893799271242 = (406.0050471772357 - 497.4694330250045) / 406.0050471772357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kotlarmilos
fanyang-mono
Run Information
Regressions in System.Numerics.Tests.Perf_Vector3
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare
Repro Steps
#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order)) - Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0` - CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0` - AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false` - Dotnet SDK installed for dotnet commands - Running commands from the runtime folder Linux ```cmd # Set $RunDir to the runtime directory RunDir=`pwd` # Set the OS, arch, and OSId RunOS='linux' RunOSId='linux' RunArch='x64' # Create aot directory mkdir -p $RunDir/artifacts/bin/aot/sgen mkdir -p $RunDir/artifacts/bin/aot/pack cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack # Create Core Root $RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance # One line run: python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector3* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200 ``` Windows ```cmd # Set $RunDir to the runtime directory $RunDir="FullPathHere" # Set the OS, arch, and OSId RunOS='windows' RunOSId='win' RunArch='x64' # Create aot directory mkdir $RunDir\artifacts\bin\aot\sgen mkdir $RunDir\artifacts\bin\aot\pack xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y # Create Core Root $RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release # Clone performance git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance # One line run: python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog" # Individual Commands: # Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Build dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 # Run dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector3* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200 ```