performanceautofiler[bot]
commented
1 year ago Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8 |
| Compare | eb71e484d2d4a7ae002ea524d2edead773289bbb |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Tests.Perf_Int64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ParseSpan(value%3a%20%229223372036854775807%22).html>) | 55.22 ns | 69.70 ns | 1.26 | 0.47 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.Parse(value%3a%20%229223372036854775807%22).html>) | 57.51 ns | 76.73 ns | 1.33 | 0.52 | False | |||||
| [TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParseSpan(value%3a%20%229223372036854775807%22).html>) | 44.96 ns | 66.63 ns | 1.48 | 0.46 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ToString(value%3a%209223372036854775807).html>) | 76.93 ns | 162.95 ns | 2.12 | 0.26 | False | |||||
| [TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParseSpan(value%3a%20%22-9223372036854775808%22).html>) | 45.43 ns | 61.55 ns | 1.35 | 0.49 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParse(value%3a%20%2212345%22).html>) | 32.64 ns | 41.80 ns | 1.28 | 0.61 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParse(value%3a%20%22-9223372036854775808%22).html>) | 47.62 ns | 60.95 ns | 1.28 | 0.49 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryFormat(value%3a%209223372036854775807).html>) | 40.58 ns | 131.95 ns | 3.25 | 0.42 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ParseSpan(value%3a%20%22-9223372036854775808%22).html>) | 52.85 ns | 64.55 ns | 1.22 | 0.49 | False | |||||
| [TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParseSpan(value%3a%20%2212345%22).html>) | 32.81 ns | 56.83 ns | 1.73 | 0.64 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.Parse(value%3a%20%2212345%22).html>) | 43.08 ns | 66.95 ns | 1.55 | 0.66 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryFormat(value%3a%20-9223372036854775808).html>) | 59.27 ns | 162.65 ns | 2.74 | 0.35 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ParseSpan(value%3a%20%2212345%22).html>) | 42.28 ns | 54.96 ns | 1.30 | 0.61 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParse(value%3a%20%229223372036854775807%22).html>) | 52.74 ns | 65.60 ns | 1.24 | 0.50 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.69722271486418 > 57.21376965748717.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.618207218009412 (T) = (0 -73.77701985020147) / Math.Sqrt((6.090305219241666 / (34)) + (33.28948933101279 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.3454474764015483 = (54.83455961247999 - 73.77701985020147) / 54.83455961247999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.Parse(value: "9223372036854775807")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.73171518186058 > 55.277100696780344.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -30.226111230195837 (T) = (0 -76.28259900625312) / Math.Sqrt((13.948352271362328 / (35)) + (0.4034106783845245 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.44296782007389074 = (52.865072903945205 - 76.28259900625312) / 52.865072903945205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.63073954863064 > 47.83851451430939.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -5.406616255749159 (T) = (0 -63.484680587041545) / Math.Sqrt((3.4722584137519847 / (34)) + (19.79537397959015 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.3685694922533897 = (46.387619296198224 - 63.484680587041545) / 46.387619296198224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 162.94584729237002 > 84.90852046412489.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -71.72020619089258 (T) = (0 -162.4125360275298) / Math.Sqrt((13.980886397797528 / (31)) + (4.952013141110234 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.9955652046573925 = (81.38673476991873 - 162.4125360275298) / 81.38673476991873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.546455230423916 > 51.119506187513295.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -8.250657081815815 (T) = (0 -63.254474546448826) / Math.Sqrt((9.643788745984606 / (33)) + (5.8346599678284 / (2))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .025) and -0.30493687391320345 = (48.47320649064296 - 63.254474546448826) / 48.47320649064296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryParse(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.79561784889793 > 32.614808339873.
IsChangePoint: Marked as a change because one of 2/16/2023 4:50:02 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.049903234718602 (T) = (0 -45.141756978475684) / Math.Sqrt((5.121644456063195 / (34)) + (22.393294148982736 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.4331006525219674 = (31.49936251793293 - 45.141756978475684) / 31.49936251793293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.949454279112 > 49.33380129582327.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -18.07128701549611 (T) = (0 -60.380531928260964) / Math.Sqrt((6.391671647542529 / (35)) + (0.6473452825957384 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.2705791541918985 = (47.52205459144622 - 60.380531928260964) / 47.52205459144622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.9461806002341 > 44.09204420183327.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -55.1799239259134 (T) = (0 -128.25094990260462) / Math.Sqrt((4.787565266136085 / (30)) + (13.43112681311448 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -1.9964961519232396 = (42.80030522324862 - 128.25094990260462) / 42.80030522324862 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.55211709502322 > 55.3456029073161.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -22.549798627119227 (T) = (0 -64.83935267687515) / Math.Sqrt((4.161665308885007 / (35)) + (0.1650085589636347 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.1849445130536705 = (54.719315514428935 - 64.83935267687515) / 54.719315514428935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryParseSpan(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.83344816601709 > 32.80146193051452.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -3.8102343706021444 (T) = (0 -51.57052420749627) / Math.Sqrt((4.997363300631069 / (33)) + (55.39673718634497 / (2))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .025) and -0.6390948077841161 = (31.46280737550148 - 51.57052420749627) / 31.46280737550148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.Parse(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.95099927281598 > 39.30962340642522.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -2.9985599645345475 (T) = (0 -59.42400855756332) / Math.Sqrt((9.373807541530855 / (35)) + (113.31117845499945 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.6147568856721796 = (36.80059152237441 - 59.42400855756332) / 36.80059152237441 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 162.6513055221099 > 63.61616699551488.
IsChangePoint: Marked as a change because one of 2/12/2023 9:12:37 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -34.9560368121235 (T) = (0 -151.0113922700575) / Math.Sqrt((5.930676250708087 / (31)) + (39.14556095990611 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -1.4991605697491839 = (60.42484588543786 - 151.0113922700575) / 60.42484588543786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.ParseSpan(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.960289614755055 > 42.2816624516581.
IsChangePoint: Marked as a change because one of 3/1/2023 8:53:57 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -6.568839189320613 (T) = (0 -52.94196263412232) / Math.Sqrt((7.227773904837821 / (35)) + (8.147287601500143 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.34534357351112854 = (39.3520017313885 - 52.94196263412232) / 39.3520017313885 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.60082218581402 > 50.085169253952266.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.914193585311775 (T) = (0 -62.51009985221211) / Math.Sqrt((7.306976939386258 / (35)) + (19.105129086851317 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.32558487732712604 = (47.15661812486561 - 62.51009985221211) / 47.15661812486561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8 |
| Compare | eb71e484d2d4a7ae002ea524d2edead773289bbb |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Tests.Perf_UInt32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt32.ParseSpan(value%3a%20%2212345%22).html>) | 34.93 ns | 41.23 ns | 1.18 | 0.60 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt32.Parse(value%3a%20%224294967295%22).html>) | 40.80 ns | 57.40 ns | 1.41 | 0.52 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt32.Parse(value%3a%20%220%22).html>) | 28.62 ns | 49.45 ns | 1.73 | 0.66 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt32.TryParse(value%3a%20%220%22).html>) | 24.40 ns | 32.53 ns | 1.33 | 0.71 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_UInt32.ParseSpan(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.227514540350086 > 37.57830111983787.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -2.68291695461199 (T) = (0 -43.19462960380771) / Math.Sqrt((10.673781791641845 / (35)) + (7.739083345763776 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.14535124569599586 = (37.712998319183406 - 43.19462960380771) / 37.712998319183406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_UInt32.Parse(value: "4294967295")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.40054780970978 > 42.12794342851184.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -15.279489154094982 (T) = (0 -58.4055504096564) / Math.Sqrt((8.326309181664977 / (35)) + (2.020060451798952 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.4129207666515904 = (41.33674851992498 - 58.4055504096564) / 41.33674851992498 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_UInt32.Parse(value: "0")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.450351313108186 > 32.2705462947169.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -8.746672932216395 (T) = (0 -47.62623966012406) / Math.Sqrt((5.118538092117825 / (35)) + (6.654766645104958 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.5204159865348607 = (31.32447966998015 - 47.62623966012406) / 31.32447966998015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### System.Tests.Perf_UInt32.TryParse(value: "0")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.53428659413416 > 26.438993506253503.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -13.682498806567938 (T) = (0 -32.61089925919735) / Math.Sqrt((7.122311435346274 / (35)) + (0.011739000896169402 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.23759162406308706 = (26.350290859382053 - 32.61089925919735) / 26.350290859382053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8 |
| Compare | eb71e484d2d4a7ae002ea524d2edead773289bbb |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2040).html>) | 17.82 ms | 19.37 ms | 1.09 | 0.13 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2010).html>) | 3.65 ms | 4.18 ms | 1.15 | 0.11 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2020).html>) | 8.03 ms | 8.89 ms | 1.11 | 0.13 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2020).html>) | 32.42 ms | 38.58 ms | 1.19 | 0.11 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%208%2c%20KeysCountPerProvider%3a%2010).html>) | 1.18 ms | 1.31 ms | 1.11 | 0.11 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2010).html>) | 15.26 ms | 17.32 ms | 1.13 | 0.14 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2040).html>) | 71.99 ms | 82.94 ms | 1.15 | 0.17 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 40)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.37450000649351 > 18.803726669583334.
IsChangePoint: Marked as a change because one of 3/9/2023 8:36:25 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -5.845330871395292 (T) = (0 -19681386.376623377) / Math.Sqrt((244178463775.31573 / (35)) + (188358488342.97543 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.1043136617004161 = (17822279.175934568 - 19681386.376623377) / 17822279.175934568 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 10)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.182601283333334 > 3.8621493336779174.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.791970092696589 (T) = (0 -4097259.113251366) / Math.Sqrt((6349194385.912121 / (34)) + (14566571988.598932 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.1124508052394767 = (3683092.406382277 - 4097259.113251366) / 3683092.406382277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 20)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.890867341836737 > 8.470271644596773.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -3.555926711192121 (T) = (0 -9219840.267072216) / Math.Sqrt((32391646129.38814 / (35)) + (216446371075.97507 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.14602711458168346 = (8045045.487809066 - 9219840.267072216) / 8045045.487809066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 20)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.58254163888889 > 34.08206858642857.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.739307005456093 (T) = (0 -37522651.748015866) / Math.Sqrt((390995865712.9618 / (33)) + (2246733161549.6387 / (2))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .025) and -0.15549996916383427 = (32473087.62385234 - 37522651.748015866) / 32473087.62385234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 10)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3100710621279763 > 1.2358503904493343.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -3.9280772196325535 (T) = (0 -1352831.1032366073) / Math.Sqrt((500499359.0271783 / (35)) + (3656842231.223617 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.1423904284517388 = (1184210.8175486685 - 1352831.1032366073) / 1184210.8175486685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.32097143333333 > 15.751684993125.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -9.078953141387247 (T) = (0 -17096166.67142857) / Math.Sqrt((114686447785.86005 / (34)) + (101074361950.11275 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.14064468859163523 = (14988161.38138281 - 17096166.67142857) / 14988161.38138281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 40)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.93911540384616 > 76.17443443.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -8.871797645850476 (T) = (0 -81843541.04358974) / Math.Sqrt((3719164507197.308 / (33)) + (2400566357702.4893 / (2))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .025) and -0.14182644527644003 = (71677741.72875734 - 81843541.04358974) / 71677741.72875734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik
Run Information
Regressions in System.Tests.Perf_Int32
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare