performanceautofiler[bot]
commented
2 years ago Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 77bf509ab733114a04936cc6688c369f0dd2c090 |
| Compare | 5ab8349289e8e463ac4ec74c0cd1ac69d52b45ba |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20None).html>) | 69.33 ms | 22.33 ms | 0.32 | 0.25 | False | |||||
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%20Holmes%22%2c%20Options%3a%20NonBacktracking).html>) | 63.29 ms | 17.32 ms | 0.27 | 0.39 | False | |||||
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Holmes%22%2c%20Options%3a%20Compiled).html>) | 130.70 ms | 41.21 ms | 0.32 | 0.44 | False | |||||
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-q%5d%5b%5eu-z%5d%7b13%7dx%22%2c%20Options%3a%20Compiled).html>) | 56.71 ms | 12.55 ms | 0.22 | 0.41 | False | |||||
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cStreet%22%2c%20Options%3a%20None).html>) | 60.65 ms | 14.39 ms | 0.24 | 0.17 | False | |||||
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%20Holmes%22%2c%20Options%3a%20None).html>) | 64.27 ms | 17.24 ms | 0.27 | 0.38 | False | |||||
| [Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%20Holmes%22%2c%20Options%3a%20Compiled).html>) | 63.68 ms | 17.32 ms | 0.27 | 0.32 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/4/2022 1:26:16 AM, 9/22/2022 1:28:19 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 2.3347893360825243 (T) = (0 -21250537.202982206) / Math.Sqrt((77279173538649.47 / (192)) + (881213484859.9701 / (3))) is greater than 1.9723316757956526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (192) + (3) - 2, .975) and 0.08397722595615911 = (23198699.64495572 - 21250537.202982206) / 23198699.64495572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/4/2022 11:58:08 PM, 9/21/2022 1:03:30 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 9.576739422798662 (T) = (0 -17227001.32202381) / Math.Sqrt((326489593170619.94 / (229)) + (15692119417.441458 / (4))) is greater than 1.970286659282671 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (4) - 2, .975) and 0.39928891035491637 = (28677681.532734793 - 17227001.32202381) / 28677681.532734793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/13/2022 1:36:49 AM, 9/19/2022 1:55:13 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 8.628444587405337 (T) = (0 -40818814.09761905) / Math.Sqrt((474625253229334.5 / (195)) + (1108648349847.2852 / (6))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (6) - 2, .975) and 0.25488421789723575 = (54781840.72604898 - 40818814.09761905) / 54781840.72604898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/13/2022 1:36:49 AM, 9/19/2022 1:55:13 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 9.130001695962294 (T) = (0 -12529483.489837093) / Math.Sqrt((105287533874292.39 / (195)) + (1452419771.4259253 / (6))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (6) - 2, .975) and 0.34877076906291504 = (19239743.69487042 - 12529483.489837093) / 19239743.69487042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/4/2022 1:26:16 AM, 9/22/2022 1:28:19 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.813828985210124 (T) = (0 -14697385.24175465) / Math.Sqrt((64174889167434.53 / (192)) + (86245797025.36234 / (3))) is greater than 1.9723316757956526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (192) + (3) - 2, .975) and 0.16481041821745815 = (17597663.527346782 - 14697385.24175465) / 17597663.527346782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/4/2022 11:58:08 PM, 9/21/2022 1:03:30 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 10.421177759221932 (T) = (0 -17253315.491391942) / Math.Sqrt((346962217208120.94 / (228)) + (320084965.6810032 / (4))) is greater than 1.9703317732749426 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (4) - 2, .975) and 0.4269752533678472 = (30109197.88856436 - 17253315.491391942) / 30109197.88856436 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: Compiled)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/18/2021 11:57:31 PM, 8/4/2022 11:58:08 PM, 9/21/2022 1:03:30 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 8.94919299723728 (T) = (0 -17334524.983163267) / Math.Sqrt((269312149725725.22 / (226)) + (6417856939.613384 / (4))) is greater than 1.9704231946743616 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (226) + (4) - 2, .975) and 0.36059191347687314 = (27110268.61956381 - 17334524.983163267) / 27110268.61956381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 77bf509ab733114a04936cc6688c369f0dd2c090 |
| Compare | 5ab8349289e8e463ac4ec74c0cd1ac69d52b45ba |
| Diff | Diff |
Improvements in StoreBlock.AnyLocation
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [InitBlockAllOnes128 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/StoreBlock.AnyLocation.InitBlockAllOnes128.html>) | 69.91 ns | 21.77 ns | 0.31 | 0.14 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'StoreBlock.AnyLocation*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### StoreBlock.AnyLocation.InitBlockAllOnes128
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/21/2022 1:44:30 AM, 1/24/2022 2:09:41 AM, 9/16/2022 1:47:08 AM, 9/22/2022 1:28:19 AM, 9/27/2022 12:44:07 AM falls between 9/18/2022 12:11:12 AM and 9/27/2022 12:44:07 AM.
IsImprovementStdDev: Marked as improvement because 5.638097999157824 (T) = (0 -21.74336087399463) / Math.Sqrt((174.05729609517826 / (218)) + (0.001228035153666191 / (3))) is greater than 1.9708553671592532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (218) + (3) - 2, .975) and 0.18815235257306548 = (26.78256313596266 - 21.74336087399463) / 26.78256313596266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kotlarmilos
Run Information
Regressions in System.Tests.Perf_DateTime
Test Report
Repro
Run Information
Regressions in System.Collections.CtorDefaultSize<String>
Test Report
Repro
Run Information
Regressions in System.IO.Tests.StreamReaderReadToEndTests
Test Report
Repro