performanceautofiler[bot]
commented
1 year ago Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
| Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
| Diff | Diff |
Improvements in System.Reflection.Attributes
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [GetCustomAttributesMethodOverrideHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit.html>) | 19.01 μs | 16.84 μs | 0.89 | 0.04 | False | |||||
| [IsDefinedMethodOverrideMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit.html>) | 1.84 μs | 1.54 μs | 0.84 | 0.03 | False | |||||
| [IsDefinedClassHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedClassHitInherit.html>) | 1.19 μs | 1.10 μs | 0.92 | 0.05 | False | |||||
| [IsDefinedMethodOverrideHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit.html>) | 1.78 μs | 1.53 μs | 0.86 | 0.06 | False | |||||
| [IsDefinedMethodBaseHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodBaseHit.html>) | 1.79 μs | 1.57 μs | 0.88 | 0.06 | False | |||||
| [GetCustomAttributesMethodOverrideMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit.html>) | 19.33 μs | 16.93 μs | 0.88 | 0.02 | False | |||||
| [GetCustomAttributesMethodOverrideHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit.html>) | 4.18 μs | 3.36 μs | 0.80 | 0.09 | False | |||||
| [IsDefinedMethodOverrideHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodOverrideHit.html>) | 1.80 μs | 1.58 μs | 0.87 | 0.07 | False | |||||
| [IsDefinedMethodBaseMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodBaseMiss.html>) | 2.14 μs | 1.76 μs | 0.82 | 0.05 | False | |||||
| [IsDefinedMethodOverrideMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodOverrideMiss.html>) | 1.79 μs | 1.59 μs | 0.88 | 0.05 | False | |||||
| [IsDefinedMethodBaseHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedMethodBaseHitInherit.html>) | 1.84 μs | 1.57 μs | 0.85 | 0.05 | False | |||||
| [GetCustomAttributesMethodOverrideMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss.html>) | 4.08 μs | 3.38 μs | 0.83 | 0.11 | False | |||||
| [GetCustomAttributesMethodBaseHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.GetCustomAttributesMethodBaseHit.html>) | 4.21 μs | 3.38 μs | 0.80 | 0.09 | False | |||||
| [IsDefinedClassMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Attributes.IsDefinedClassMiss.html>) | 1.31 μs | 1.11 μs | 0.85 | 0.06 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.84175635091496 < 18.11997691771894.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 30.712255299955377 (T) = (0 -16762.028651648372) / Math.Sqrt((444049.765092124 / (299)) + (66159.29977183756 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1329075479341004 = (19331.304997191288 - 16762.028651648372) / 19331.304997191288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.542388696045254 < 1.7231218711399645.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 38.93019940898805 (T) = (0 -1568.3658689184897) / Math.Sqrt((1228.9004449796707 / (299)) + (457.26607647996394 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.13887983344889032 = (1821.3089529652807 - 1568.3658689184897) / 1821.3089529652807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedClassHitInherit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.100935367120688 < 1.1490428740026346.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 33.3306903960601 (T) = (0 -1104.4225965727057) / Math.Sqrt((1587.2269553895228 / (299)) + (129.40227019844122 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.10798997052086906 = (1238.128003131992 - 1104.4225965727057) / 1238.128003131992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5294606565227529 < 1.6884810616614945.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 37.86939752382803 (T) = (0 -1538.746310826694) / Math.Sqrt((1693.4871662493977 / (299)) + (532.8493997157201 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1483136076192247 = (1806.7052903420645 - 1538.746310826694) / 1806.7052903420645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodBaseHit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5673449421801744 < 1.728989657900026.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 34.480448412094 (T) = (0 -1565.7295971915912) / Math.Sqrt((1352.5449063239253 / (299)) + (626.6252746958883 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.14228321015351336 = (1825.4622221768843 - 1565.7295971915912) / 1825.4622221768843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.929470004515228 < 18.314663755515074.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 38.62341758672281 (T) = (0 -16708.46833233935) / Math.Sqrt((412446.88394095644 / (299)) + (37407.7387004284 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1342073990988804 = (19298.46514621299 - 16708.46833233935) / 19298.46514621299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.359763615485564 < 3.8246415424813307.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 40.999358876513924 (T) = (0 -3309.826530730654) / Math.Sqrt((23812.7214931903 / (299)) + (3520.16898422418 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1930487251750847 = (4101.643598554063 - 3309.826530730654) / 4101.643598554063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodOverrideHit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5755093316369808 < 1.6904946703598627.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 36.661221769746106 (T) = (0 -1546.4516270165286) / Math.Sqrt((1632.581767018218 / (299)) + (553.5200074972242 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.14549684861539172 = (1809.767026032274 - 1546.4516270165286) / 1809.767026032274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodBaseMiss
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7587840468559217 < 1.973589239353886.
IsChangePoint: Marked as a change because one of 1/30/2023 9:42:46 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 40.09295599095631 (T) = (0 -1750.404251941185) / Math.Sqrt((2449.872612997476 / (299)) + (552.568193839636 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.14434219090114145 = (2045.6825536187584 - 1750.404251941185) / 2045.6825536187584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodOverrideMiss
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.586553518031314 < 1.7104849274528928.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 32.166879679587765 (T) = (0 -1564.0603754174674) / Math.Sqrt((1552.0583196908458 / (299)) + (713.3801233982841 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.14188845659638089 = (1822.6772351922552 - 1564.0603754174674) / 1822.6772351922552 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedMethodBaseHitInherit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.56687009529148 < 1.7303771950595033.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 30.735579857711844 (T) = (0 -1567.4110844235938) / Math.Sqrt((1554.9208150678621 / (299)) + (798.7210205271443 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.14244949088737194 = (1827.7769854576984 - 1567.4110844235938) / 1827.7769854576984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.375562298213517 < 3.9005907547710277.
IsChangePoint: Marked as a change because one of 2/1/2023 6:08:01 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 25.111249645824994 (T) = (0 -3399.5098522824082) / Math.Sqrt((26351.05926355904 / (299)) + (10348.566541738288 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.18549228045448846 = (4173.698751657389 - 3399.5098522824082) / 4173.698751657389 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHit
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.3787814287268434 < 3.9495169757540407.
IsChangePoint: Marked as a change because one of 2/1/2023 6:08:01 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 44.343626184895776 (T) = (0 -3385.8760505923547) / Math.Sqrt((31896.015676933617 / (299)) + (2616.1832293907614 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1909325387778615 = (4184.911905217165 - 3385.8760505923547) / 4184.911905217165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedClassMiss
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1129562704757712 < 1.2425658045779977.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 23.792551783611763 (T) = (0 -1141.681352336444) / Math.Sqrt((889.154144501026 / (299)) + (520.953745979925 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.12429410405441031 = (1303.7269220434484 - 1141.681352336444) / 1303.7269220434484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
| Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
| Diff | Diff |
Improvements in PerfLabTests.LowLevelPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [NewDelegateClassEmptyInstanceFn - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.NewDelegateClassEmptyInstanceFn.html>) | 23.27 ms | 18.69 ms | 0.80 | 0.08 | False | |||||
| [NewDelegateClassEmptyStaticFn - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.NewDelegateClassEmptyStaticFn.html>) | 18.80 ms | 14.77 ms | 0.79 | 0.15 | False | |||||
| [GenericClassWithSTringGenericInstanceMethod - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.GenericClassWithSTringGenericInstanceMethod.html>) | 1.72 ms | 1.31 ms | 0.76 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### PerfLabTests.LowLevelPerf.NewDelegateClassEmptyInstanceFn
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.69257948205128 < 21.570451791937227.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 71.95695151136329 (T) = (0 -18476025.026195172) / Math.Sqrt((484415104179.7195 / (299)) + (31030883626.218407 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.20164822441086125 = (23142711.760816094 - 18476025.026195172) / 23142711.760816094 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.NewDelegateClassEmptyStaticFn
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.7700958375 < 16.90865982869898.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 19.182537878744018 (T) = (0 -14825667.526376994) / Math.Sqrt((351114026501.95605 / (299)) + (268877395611.98096 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1657542150044098 = (17771342.442509748 - 14825667.526376994) / 17771342.442509748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.GenericClassWithSTringGenericInstanceMethod
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3068679895833333 < 1.636822066380952.
IsChangePoint: Marked as a change because one of 1/22/2023 3:25:37 PM, 1/22/2023 11:22:03 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 61.57550035961432 (T) = (0 -1310448.6781858911) / Math.Sqrt((12854456777.700727 / (299)) + (96233425.95374998 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.25127182719681485 = (1750232.8959783418 - 1310448.6781858911) / 1750232.8959783418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
| Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
| Diff | Diff |
Improvements in System.Tests.Perf_Environment
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [GetLogicalDrives - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Environment.GetLogicalDrives.html>) | 53.53 ns | 39.49 ns | 0.74 | 0.19 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_Environment.GetLogicalDrives
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.485318514774725 < 50.943755948260545.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 1/20/2023 8:02:54 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 55.850629509225385 (T) = (0 -40.02785759609197) / Math.Sqrt((5.776406498384679 / (299)) + (0.569559673129355 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2650171212313092 = (54.46093882234403 - 40.02785759609197) / 54.46093882234403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_9.png>)
_10.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_1.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_1.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_1.png>)
_1.png>)
_1.png>)
_1.png>)
_1.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_2.png>)
_3.png>)
_1.png>)
_2.png>)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_1.png>)
_1.png>)
_1.png>)
_1.png>)
radekdoulik
lewing
Run Information
Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare
Run Information
Improvements in System.Diagnostics.Perf_Activity
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline Compare