performanceautofiler[bot]
commented
2 years ago Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 6b8d056a665fa20d321ea1d49a93a37a83960663 |
| Compare | 59616049ef3f6f95d0e8943db446b5311a9cd141 |
| Diff | Diff |
Regressions in System.Formats.Cbor.Tests.Perf_CborReader
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ReadCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey%3a%20ECDSA_P521).html>) | 632.91 ns | 1.09 μs | 1.72 | 0.00 | False | |||||
| [SkipValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding%3a%20(Array%2c%20Strict)).html>) | 312.97 ns | 466.54 ns | 1.49 | 0.01 | False | |||||
| [ReadCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey%3a%20ECDSA_P256).html>) | 623.22 ns | 1.07 μs | 1.72 | 0.00 | False | |||||
| [SkipValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding%3a%20(Map%2c%20Lax)).html>) | 661.33 ns | 1.14 μs | 1.72 | 0.01 | False | |||||
| [ReadCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey%3a%20ECDSA_P384).html>) | 627.87 ns | 1.08 μs | 1.72 | 0.00 | False | |||||
| [SkipValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding%3a%20(Text%20String%2c%20Strict)).html>) | 67.20 ns | 113.63 ns | 1.69 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey: ECDSA_P521)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0889535326566284 > 665.8465183206365.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -132.65737882987992 (T) = (0 -1089.7650045921278) / Math.Sqrt((11.765806719294897 / (39)) + (57.92724479707406 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.7232414501549762 = (632.3925207893078 - 1089.7650045921278) / 632.3925207893078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Array, Strict))
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.5359706500204 > 321.4682308014141.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -78.24703498105899 (T) = (0 -468.7833667147567) / Math.Sqrt((134.8017741914312 / (43)) + (14.717190381446807 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.6180985817877265 = (289.71248846706857 - 468.7833667147567) / 289.71248846706857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey: ECDSA_P256)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0742104200857643 > 656.1564959513361.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -292.3271941940449 (T) = (0 -1072.9914800495617) / Math.Sqrt((8.77878480050338 / (40)) + (12.955831325923697 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.7247141453579946 = (622.1271408583731 - 1072.9914800495617) / 622.1271408583731 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Map, Lax))
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1406195758298983 > 694.0804510452089.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -132.54684029027663 (T) = (0 -1130.6685319504734) / Math.Sqrt((52.57923981153919 / (45)) + (66.9646442401212 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.6995958176719755 = (665.2573042331962 - 1130.6685319504734) / 665.2573042331962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey: ECDSA_P384)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0790472179309614 > 658.7048030096287.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -144.89878168817228 (T) = (0 -1082.704421003379) / Math.Sqrt((9.721020861654216 / (40)) + (48.19687920822504 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.7262604331417445 = (627.1964532216543 - 1082.704421003379) / 627.1964532216543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict))
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.63470406274077 > 70.74300668029872.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -113.7939491327966 (T) = (0 -112.60415466580845) / Math.Sqrt((0.4298536103626667 / (44)) + (1.0373865357526186 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.6712647679930397 = (67.37661011129352 - 112.60415466580845) / 67.37661011129352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 6b8d056a665fa20d321ea1d49a93a37a83960663 |
| Compare | 59616049ef3f6f95d0e8943db446b5311a9cd141 |
| Diff | Diff |
Regressions in System.Tests.Perf_UInt32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.TryFormat(value%3a%200).html>) | 2.37 ns | 16.92 ns | 7.15 | 0.02 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.TryParse(value%3a%20%2212345%22).html>) | 16.27 ns | 27.01 ns | 1.66 | 0.10 | False | |||||
| [TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.TryParseHex(value%3a%20%223039%22).html>) | 10.23 ns | 78.85 ns | 7.71 | 0.01 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.TryFormat(value%3a%204294967295).html>) | 14.76 ns | 25.85 ns | 1.75 | 0.02 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.ParseSpan(value%3a%20%224294967295%22).html>) | 19.49 ns | 34.34 ns | 1.76 | 0.08 | False | |||||
| [TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.TryParse(value%3a%20%220%22).html>) | 10.34 ns | 22.89 ns | 2.21 | 0.22 | False | |||||
| [ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.ParseSpan(value%3a%20%220%22).html>) | 10.26 ns | 24.20 ns | 2.36 | 0.24 | False | |||||
| [Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.Parse(value%3a%20%220%22).html>) | 10.17 ns | 24.44 ns | 2.40 | 0.19 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt32.TryFormat(value%3a%2012345).html>) | 8.59 ns | 20.95 ns | 2.44 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_UInt32.TryFormat(value: 0)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.91966495575429 > 2.4776547487598863.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -390.6845353118678 (T) = (0 -16.861869582184724) / Math.Sqrt((5.086078003709308E-06 / (40)) + (0.008267305653136816 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -6.147756066738653 = (2.359043792869443 - 16.861869582184724) / 2.359043792869443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.TryParse(value: "12345")
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.010873390959024 > 17.29571142811355.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -49.42967381524521 (T) = (0 -26.072299540229515) / Math.Sqrt((0.12654280588133346 / (44)) + (0.28440900872015357 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.6540993910206176 = (15.762232718157463 - 26.072299540229515) / 15.762232718157463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.TryParseHex(value: "3039")
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.8535215333743 > 10.729882202014581.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -143.11602249285013 (T) = (0 -79.47503623962695) / Math.Sqrt((0.04751503916877666 / (38)) + (1.167333953315415 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -6.838756717349125 = (10.138729789091279 - 79.47503623962695) / 10.138729789091279 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.847544485776115 > 15.524049989489571.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -77.16966225120265 (T) = (0 -25.742489729917178) / Math.Sqrt((0.28994006014829105 / (48)) + (0.12089953596548741 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.844008031707353 = (13.960074624014736 - 25.742489729917178) / 13.960074624014736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.ParseSpan(value: "4294967295")
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.34486911494328 > 20.68407522808782.
IsChangePoint: Marked as a change because one of 6/2/2022 5:58:32 PM, 6/6/2022 7:24:37 AM, 6/27/2022 8:54:23 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -31.036527637527342 (T) = (0 -34.65600785426642) / Math.Sqrt((3.637291612581206 / (43)) + (0.5028555419762483 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.5484644325968664 = (22.38088723558619 - 34.65600785426642) / 22.38088723558619 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.TryParse(value: "0")
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.891623737833722 > 10.836116959349829.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -66.79254435565488 (T) = (0 -22.564025451643634) / Math.Sqrt((0.31640947388295937 / (46)) + (0.19340964427110405 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -1.2216672530626114 = (10.156347860166138 - 22.564025451643634) / 10.156347860166138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.ParseSpan(value: "0")
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.19515921370389 > 10.668977919001438.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -111.67145405317714 (T) = (0 -24.39245559487208) / Math.Sqrt((0.1706682760915452 / (44)) + (0.08652266570762958 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -1.400404255307521 = (10.16181151193098 - 24.39245559487208) / 10.16181151193098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.Parse(value: "0")
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.43579370204875 > 10.680173444642229.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.80076226331446 (T) = (0 -23.993993093375718) / Math.Sqrt((0.1796345195499972 / (45)) + (0.3197362745316433 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -1.3615563519986056 = (10.160245836636248 - 23.993993093375718) / 10.160245836636248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.94880127893083 > 9.009087211079217.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -80.60169656550127 (T) = (0 -20.625262976970596) / Math.Sqrt((0.013012744476194951 / (40)) + (0.11315158760370954 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -1.4515345124969241 = (8.413205227922107 - 20.625262976970596) / 8.413205227922107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 6b8d056a665fa20d321ea1d49a93a37a83960663 |
| Compare | 59616049ef3f6f95d0e8943db446b5311a9cd141 |
| Diff | Diff |
Regressions in System.Collections.ContainsTrue<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.ContainsTrue(String).SortedSet(Size%3a%20512).html>) | 247.63 μs | 336.41 μs | 1.36 | 0.29 | False | |||||
| [ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.ContainsTrue(String).ImmutableSortedSet(Size%3a%20512).html>) | 255.28 μs | 340.12 μs | 1.33 | 0.31 | False |
_1.png>)
_2.png>)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<String>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Collections.ContainsTrue<String>.SortedSet(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 336.40730496453904 > 257.54052164713545.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -39.954045031818815 (T) = (0 -336429.1420656028) / Math.Sqrt((70418269.09835629 / (38)) + (14256887.769381447 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.3469586081930806 = (249769.4732556898 - 336429.1420656028) / 249769.4732556898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsTrue<String>.ImmutableSortedSet(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 340.1188768115942 > 265.13153186603944.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -23.980130684752044 (T) = (0 -341099.542012666) / Math.Sqrt((92840617.62369713 / (38)) + (70405047.00872786 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.33079061196403614 = (256313.45678735824 - 341099.542012666) / 256313.45678735824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_9.png>)
_10.png>)
_11.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_2.png>)
_1.png>)
_1.png>)
_1.png>)
_2.png>)
_1.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_1.png>)
_1.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_1.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_1.png>)
_1.png>)
_1.png>)
_2.png>)
_1.png>)
_1.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_9.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
kunalspathak
Run Information
Regressions in MicroBenchmarks.Serializers.Binary_ToStream<Location>
Repro