performanceautofiler[bot]
commented
1 year ago Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8 |
| Compare | eb71e484d2d4a7ae002ea524d2edead773289bbb |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%208%2c%20KeysCountPerProvider%3a%2040).html>) | 5.28 ms | 5.77 ms | 1.09 | 0.16 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2040).html>) | 17.75 ms | 19.14 ms | 1.08 | 0.14 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2010).html>) | 14.65 ms | 17.04 ms | 1.16 | 0.14 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%208%2c%20KeysCountPerProvider%3a%2010).html>) | 1.14 ms | 1.25 ms | 1.09 | 0.11 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2020).html>) | 7.70 ms | 8.72 ms | 1.13 | 0.13 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2040).html>) | 72.13 ms | 87.68 ms | 1.22 | 0.18 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2016%2c%20KeysCountPerProvider%3a%2010).html>) | 3.70 ms | 4.05 ms | 1.10 | 0.12 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%2032%2c%20KeysCountPerProvider%3a%2020).html>) | 32.23 ms | 36.14 ms | 1.12 | 0.12 | False | |||||
| [Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount%3a%208%2c%20KeysCountPerProvider%3a%2020).html>) | 2.45 ms | 2.69 ms | 1.10 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 40)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.768193198568874 > 5.555190852553191.
IsChangePoint: Marked as a change because one of 3/10/2023 8:23:08 AM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -13.132387278711654 (T) = (0 -5869651.31638145) / Math.Sqrt((17804121861.134914 / (15)) + (28041631669.697735 / (33))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (33) - 2, .025) and -0.11231029679543837 = (5276990.90199191 - 5869651.31638145) / 5276990.90199191 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 40)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.136672763636362 > 18.31078040208334.
IsChangePoint: Marked as a change because one of 3/9/2023 8:36:25 AM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -12.386883308895136 (T) = (0 -19956375.88218499) / Math.Sqrt((315567668746.24146 / (16)) + (306591793264.3837 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.11822541171997035 = (17846469.65900157 - 19956375.88218499) / 17846469.65900157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.044040795918363 > 15.6750409265625.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -22.324647609553768 (T) = (0 -17156226.4847026) / Math.Sqrt((55936659505.39759 / (15)) + (217468604642.39606 / (33))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (33) - 2, .025) and -0.15231969878070675 = (14888425.931497965 - 17156226.4847026) / 14888425.931497965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 10)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2492368857868017 > 1.2356239850274726.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -13.658132588630975 (T) = (0 -1289371.1886930068) / Math.Sqrt((550878775.1354471 / (16)) + (1018630969.9682293 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.09361046260248634 = (1179004.072094066 - 1289371.1886930068) / 1179004.072094066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 20)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.723905609693878 > 8.42879366923387.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -20.517249067463112 (T) = (0 -9080784.890148683) / Math.Sqrt((24419601647.97819 / (16)) + (38847134521.79113 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.1331145737977953 = (8014004.144093863 - 9080784.890148683) / 8014004.144093863 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 40)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.67663329999999 > 74.50538431875.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -17.198955521198013 (T) = (0 -82305803.98281303) / Math.Sqrt((3884257173451.6416 / (14)) + (5602148313355.922 / (33))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (33) - 2, .025) and -0.16244193467803983 = (70804228.17472528 - 82305803.98281303) / 70804228.17472528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 10)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.048785620428752 > 3.866244357395362.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.83538653738699 (T) = (0 -4143407.7782462104) / Math.Sqrt((6343966639.785733 / (16)) + (11667832531.617731 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.1252133980955963 = (3682330.6452436973 - 4143407.7782462104) / 3682330.6452436973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 20)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.14403572619048 > 33.891899994999996.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -19.390571897821093 (T) = (0 -37156363.10415603) / Math.Sqrt((471584803094.6791 / (16)) + (1004663942371.0043 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.14645127973987257 = (32409892.823868394 - 37156363.10415603) / 32409892.823868394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 20)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6853139784946234 > 2.585603311949115.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -10.545807702491079 (T) = (0 -2713841.1656068875) / Math.Sqrt((4954062603.603695 / (16)) + (2019854236.0023983 / (32))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (32) - 2, .025) and -0.0811098701609641 = (2510236.230849349 - 2713841.1656068875) / 2510236.230849349 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8 |
| Compare | eb71e484d2d4a7ae002ea524d2edead773289bbb |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Tests.Perf_UInt64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.Parse(value%3a%20%2218446744073709551615%22).html>) | 49.24 ns | 69.39 ns | 1.41 | 0.58 | False | |||||
| [TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParseHex(value%3a%20%223039%22).html>) | 29.57 ns | 41.76 ns | 1.41 | 0.64 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParse(value%3a%20%2212345%22).html>) | 28.38 ns | 35.33 ns | 1.24 | 0.65 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ParseSpan(value%3a%20%2218446744073709551615%22).html>) | 57.32 ns | 62.81 ns | 1.10 | 0.52 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryFormat(value%3a%2018446744073709551615).html>) | 45.10 ns | 127.16 ns | 2.82 | 0.44 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ParseSpan(value%3a%20%2212345%22).html>) | 44.63 ns | 50.69 ns | 1.14 | 0.63 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.Parse(value%3a%20%2212345%22).html>) | 38.70 ns | 60.17 ns | 1.55 | 0.69 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParse(value%3a%20%220%22).html>) | 24.24 ns | 40.07 ns | 1.65 | 0.71 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.Parse(value%3a%20%220%22).html>) | 30.49 ns | 48.71 ns | 1.60 | 0.70 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ToString(value%3a%2018446744073709551615).html>) | 76.78 ns | 169.25 ns | 2.20 | 0.35 | False | |||||
| [TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParseHex(value%3a%20%22FFFFFFFFFFFFFFFF%22).html>) | 48.27 ns | 55.28 ns | 1.15 | 0.43 | False | |||||
| [TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParseHex(value%3a%20%220%22).html>) | 27.31 ns | 33.62 ns | 1.23 | 0.69 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ParseSpan(value%3a%20%220%22).html>) | 38.77 ns | 48.61 ns | 1.25 | 0.67 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.38543003077977 > 52.079848172224736.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -19.622382792919495 (T) = (0 -71.03184897155052) / Math.Sqrt((7.592202312241995 / (16)) + (17.825024646192126 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.38555177040464017 = (51.26610963862158 - 71.03184897155052) / 51.26610963862158 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.TryParseHex(value: "3039")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.756735382059794 > 31.38999733923391.
IsChangePoint: Marked as a change because one of 4/19/2023 4:18:46 AM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -9.154957737585704 (T) = (0 -41.84211037453903) / Math.Sqrt((9.136489572556911 / (23)) + (16.948220830691433 / (26))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (26) - 2, .025) and -0.28883270955117607 = (32.465121395863825 - 41.84211037453903) / 32.465121395863825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.TryParse(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.33098299694239 > 30.038756379790165.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -7.480547245075481 (T) = (0 -39.58713127951981) / Math.Sqrt((11.850549850792373 / (14)) + (12.105162133255783 / (33))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (33) - 2, .025) and -0.26285428301242303 = (31.34734688873869 - 39.58713127951981) / 31.34734688873869 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.805243922857166 > 60.94421773731896.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -6.873980530612744 (T) = (0 -64.9280311531611) / Math.Sqrt((13.602941090802279 / (16)) + (24.692702660869664 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.1545372611002733 = (56.23727647497901 - 64.9280311531611) / 56.23727647497901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.16284509181718 > 46.67220989952108.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -32.98584452203704 (T) = (0 -132.78559902424735) / Math.Sqrt((5.195673713900492 / (11)) + (255.70050021203957 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -2.076834023982964 = (43.15656872916281 - 132.78559902424735) / 43.15656872916281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.ParseSpan(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.692296495760715 > 44.24639431856113.
IsChangePoint: Marked as a change because one of 4/18/2023 3:11:52 AM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -7.049762320437948 (T) = (0 -49.618665602763095) / Math.Sqrt((12.917954397123518 / (17)) + (20.96811799383553 / (31))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (31) - 2, .025) and -0.20521668127614964 = (41.169912741519745 - 49.618665602763095) / 41.169912741519745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.Parse(value: "12345")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.16811701090525 > 39.83948447230291.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -9.762060071358661 (T) = (0 -57.28218409147892) / Math.Sqrt((18.821490617343898 / (16)) + (101.47997680921289 / (32))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (32) - 2, .025) and -0.5512096601725848 = (36.927428678535826 - 57.28218409147892) / 36.927428678535826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.TryParse(value: "0")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.0711883809115 > 29.118664471735592.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -9.548484655764964 (T) = (0 -36.146455817959534) / Math.Sqrt((7.769762963600331 / (16)) + (12.270731811014217 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.3238185619652308 = (27.304690277419525 - 36.146455817959534) / 27.304690277419525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.Parse(value: "0")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.71294987686317 > 31.16359436383898.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -20.435472197627313 (T) = (0 -50.82405804101343) / Math.Sqrt((5.086012230164917 / (15)) + (19.931522446678954 / (32))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (32) - 2, .025) and -0.651130211697066 = (30.781374891551046 - 50.82405804101343) / 30.781374891551046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 169.24652896070134 > 80.55330650805246.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -98.02907785600303 (T) = (0 -174.1525998121457) / Math.Sqrt((7.068416512425327 / (12)) + (15.249254098874433 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -1.2894974583628227 = (76.0658629150342 - 174.1525998121457) / 76.0658629150342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.279927114093205 > 50.632196009860884.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -5.8090394262950165 (T) = (0 -55.854063063384736) / Math.Sqrt((10.060556578382899 / (22)) + (15.539037553130987 / (26))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (26) - 2, .025) and -0.11959934762264998 = (49.887545202652085 - 55.854063063384736) / 49.887545202652085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.TryParseHex(value: "0")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.62423647824202 > 28.40988261055646.
IsChangePoint: Marked as a change because one of 4/19/2023 4:18:46 AM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -8.85179785883588 (T) = (0 -36.82941391458567) / Math.Sqrt((9.347454933141083 / (22)) + (14.603406292642092 / (26))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (26) - 2, .025) and -0.31358496607617786 = (28.037329039018438 - 36.82941391458567) / 28.037329039018438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
#### System.Tests.Perf_UInt64.ParseSpan(value: "0")
```log
```
### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.613100197997404 > 37.51673727250351.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -7.701244476940044 (T) = (0 -43.20258116512352) / Math.Sqrt((7.012589633478467 / (16)) + (20.651497469310264 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.2253137296953209 = (35.258383316953456 - 43.20258116512352) / 35.258383316953456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### JIT Disasms
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik
Run Information
Regressions in System.Tests.Perf_Int32
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare
Run Information
Regressions in System.Tests.Perf_UInt16
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare