search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 5/9/2022 7:27:05 AM #5220

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetCustomAttributesClassMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesClassMiss.html>) 219.16 ns 187.40 ns 0.86 0.05 False
[IsDefinedMethodOverrideMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit.html>) 646.15 ns 578.19 ns 0.89 0.07 False
[IsDefinedMethodOverrideMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodOverrideMiss.html>) 651.71 ns 565.19 ns 0.87 0.08 False
[GetCustomAttributesMethodBaseHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodBaseHit.html>) 1.29 μs 1.14 μs 0.88 0.23 False
[GetCustomAttributesMethodBaseMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit.html>) 240.74 ns 207.23 ns 0.86 0.10 False
[GetCustomAttributesMethodBaseMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss.html>) 212.66 ns 178.96 ns 0.84 0.04 False
[IsDefinedClassHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedClassHitInherit.html>) 633.70 ns 562.52 ns 0.89 0.07 False
[GetCustomAttributesClassHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesClassHit.html>) 1.29 μs 1.18 μs 0.91 0.18 False
[IsDefinedClassHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedClassHit.html>) 639.28 ns 550.04 ns 0.86 0.10 False
[IsDefinedMethodOverrideHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodOverrideHit.html>) 677.92 ns 588.28 ns 0.87 0.07 False
[GetCustomAttributesMethodOverrideHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit.html>) 2.26 μs 1.95 μs 0.87 0.14 False
[GetCustomAttributesMethodOverrideMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss.html>) 1.27 μs 1.14 μs 0.90 0.19 False
[GetCustomAttributesMethodBaseHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit.html>) 1.30 μs 1.17 μs 0.90 0.14 False
[GetCustomAttributesMethodOverrideHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit.html>) 1.33 μs 1.14 μs 0.85 0.20 False
[IsDefinedMethodOverrideHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit.html>) 664.47 ns 578.03 ns 0.87 0.07 False
[IsDefinedMethodBaseMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodBaseMissInherit.html>) 165.66 ns 134.15 ns 0.81 0.15 False
[IsDefinedClassMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedClassMissInherit.html>) 1.08 μs 834.84 ns 0.78 0.07 False
[GetCustomAttributesMethodOverrideMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit.html>) 1.32 μs 1.15 μs 0.88 0.20 False
[IsDefinedClassMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedClassMiss.html>) 138.24 ns 108.25 ns 0.78 0.16 False
[IsDefinedMethodBaseHitInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodBaseHitInherit.html>) 644.72 ns 569.15 ns 0.88 0.07 False
[IsDefinedMethodBaseMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.IsDefinedMethodBaseMiss.html>) 134.47 ns 106.67 ns 0.79 0.17 False
[GetCustomAttributesClassMissInherit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Reflection.Attributes.GetCustomAttributesClassMissInherit.html>) 367.27 ns 290.72 ns 0.79 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Attributes*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Attributes.GetCustomAttributesClassMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 187.3998456486849 < 204.04536995316766. IsChangePoint: Marked as a change because one of 3/4/2022 1:10:06 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 8.130477906635658 (T) = (0 -182.722277505863) / Math.Sqrt((35.99558348935294 / (299)) + (97.33552366819497 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1428386488074278 = (213.17139095415433 - 182.722277505863) / 213.17139095415433 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 578.189402490533 < 623.4172856248983. IsChangePoint: Marked as a change because one of 3/24/2022 8:39:13 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 16.321179788181688 (T) = (0 -574.5549740454874) / Math.Sqrt((440.9110630077237 / (299)) + (220.20625747146315 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.140167462882476 = (668.2172972560537 - 574.5549740454874) / 668.2172972560537 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodOverrideMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 565.1929996955186 < 629.1933389681076. IsChangePoint: Marked as a change because one of 3/25/2022 1:38:06 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 63.30994820903242 (T) = (0 -568.0240884064975) / Math.Sqrt((356.36720236032204 / (299)) + (9.988433514843104 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1528052233118716 = (670.4763816262291 - 568.0240884064975) / 670.4763816262291 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1366408975578814 < 1.2216514683233242. IsChangePoint: Marked as a change because one of 4/11/2022 1:47:52 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 23.505299891947775 (T) = (0 -1142.2541374588743) / Math.Sqrt((3532.470790610257 / (299)) + (294.2676883534175 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13119791827504154 = (1314.7460871536955 - 1142.2541374588743) / 1314.7460871536955 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 207.22761484550153 < 233.801178583444. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 3.5241370876950255 (T) = (0 -211.35559915165518) / Math.Sqrt((42.4800919054651 / (299)) + (473.86166044792895 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12074909553885547 = (240.38144070057686 - 211.35559915165518) / 240.38144070057686 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 178.9635505248959 < 205.1092488668356. IsChangePoint: Marked as a change because one of 3/4/2022 1:10:06 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.620468490004768 (T) = (0 -178.56657948236108) / Math.Sqrt((24.799242836096898 / (299)) + (10.197541550698357 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16102785495738203 = (212.83969978918702 - 178.56657948236108) / 212.83969978918702 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedClassHitInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 562.52031259233 < 607.2434932611766. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 28.132289359556744 (T) = (0 -558.9447398049982) / Math.Sqrt((339.2906240729612 / (299)) + (45.1008066823181 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13821995044155794 = (648.5932693514891 - 558.9447398049982) / 648.5932693514891 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesClassHit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1772303884522368 < 1.2364890632495216. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 19.03639942617631 (T) = (0 -1159.9048317454976) / Math.Sqrt((3258.8740350125545 / (299)) + (292.80274317586213 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.10648442332340415 = (1298.1361064344603 - 1159.9048317454976) / 1298.1361064344603 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedClassHit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 550.0374629821904 < 609.3657788391283. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 36.21844823369917 (T) = (0 -548.4690469017221) / Math.Sqrt((370.1769406714992 / (299)) + (45.4706756240685 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15515112259659583 = (649.1918987776975 - 548.4690469017221) / 649.1918987776975 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodOverrideHit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 588.2755392178251 < 652.2829777649977. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/8/2022 10:19:59 AM, 4/19/2022 12:38:15 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 37.16103181017471 (T) = (0 -579.6948015875372) / Math.Sqrt((358.1755941611117 / (299)) + (57.19534246850981 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16402837397557266 = (693.4383698455794 - 579.6948015875372) / 693.4383698455794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.952352045315481 < 2.0399074575459983. IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 20.374338763428426 (T) = (0 -1880.9050110080473) / Math.Sqrt((5815.622922246635 / (299)) + (1359.6364311837451 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13669888812207698 = (2178.735768006309 - 1880.9050110080473) / 2178.735768006309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.138604990757856 < 1.2312535622455536. IsChangePoint: Marked as a change because one of 4/11/2022 1:47:52 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 20.825616622782885 (T) = (0 -1142.190837365773) / Math.Sqrt((3579.563717559721 / (299)) + (402.16157858472224 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1318834488195389 = (1315.7113936056476 - 1142.190837365773) / 1315.7113936056476 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1702854938271605 < 1.2708386353603103. IsChangePoint: Marked as a change because one of 3/22/2022 2:14:22 AM, 4/17/2022 3:50:42 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 18.26437712697967 (T) = (0 -1195.3978544001095) / Math.Sqrt((2816.074285703619 / (299)) + (404.938192128119 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11135745676892858 = (1345.1953921243598 - 1195.3978544001095) / 1345.1953921243598 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1408193187898246 < 1.2568735756016736. IsChangePoint: Marked as a change because one of 4/11/2022 1:47:52 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 35.87515146424362 (T) = (0 -1149.1060258236478) / Math.Sqrt((2869.882652387936 / (299)) + (144.82272429698543 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14662364942531647 = (1346.5407437759588 - 1149.1060258236478) / 1346.5407437759588 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 578.0327139283098 < 650.7271209410042. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/8/2022 7:07:28 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 17.687542850904872 (T) = (0 -586.7225818912624) / Math.Sqrt((368.669051588078 / (299)) + (242.0100809647297 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15282219094462435 = (692.561320209122 - 586.7225818912624) / 692.561320209122 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodBaseMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 134.1451924956875 < 160.2366764616816. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 29.79630881026327 (T) = (0 -130.6395415182862) / Math.Sqrt((76.15654000416484 / (299)) + (9.976642122646332 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2281690486111892 = (169.25926756787493 - 130.6395415182862) / 169.25926756787493 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedClassMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 834.8399732620322 < 1.0261226600431248. IsChangePoint: Marked as a change because one of 4/29/2022 7:06:31 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 26.279196515915967 (T) = (0 -842.8440881096737) / Math.Sqrt((1067.353914326055 / (299)) + (462.8865898255871 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2065365585208705 = (1062.23430601729 - 842.8440881096737) / 1062.23430601729 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1530139433033713 < 1.26330913951067. IsChangePoint: Marked as a change because one of 4/12/2022 1:19:04 PM, 5/9/2022 7:27:05 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 24.138611755780833 (T) = (0 -1167.5312711129764) / Math.Sqrt((3654.2115229119427 / (299)) + (263.8060749863647 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1341825997881784 = (1348.4728660192561 - 1167.5312711129764) / 1348.4728660192561 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedClassMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 108.24534276934405 < 130.9593430728011. IsChangePoint: Marked as a change because one of 4/4/2022 8:14:13 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 42.5928161074359 (T) = (0 -107.04733647354774) / Math.Sqrt((57.52212153422237 / (299)) + (4.052399430392335 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2589511452873825 = (144.4538181157587 - 107.04733647354774) / 144.4538181157587 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodBaseHitInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 569.1484720887362 < 627.9414814369744. IsChangePoint: Marked as a change because one of 3/24/2022 8:39:13 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 63.547306735424435 (T) = (0 -569.4685180708999) / Math.Sqrt((428.04453193394164 / (299)) + (6.623472605240878 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1468114346018678 = (667.4591540091292 - 569.4685180708999) / 667.4591540091292 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.IsDefinedMethodBaseMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 106.67274452381284 < 131.23006408116083. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 37.397756687911645 (T) = (0 -104.98694704655462) / Math.Sqrt((29.029746030182466 / (299)) + (5.625880396354018 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2526612832852989 = (140.4810759812859 - 104.98694704655462) / 140.4810759812859 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Attributes.GetCustomAttributesClassMissInherit ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 290.7150440012298 < 348.87078910316325. IsChangePoint: Marked as a change because one of 4/19/2022 11:38:14 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 29.757301983081675 (T) = (0 -293.2882800867366) / Math.Sqrt((159.89371859416872 / (299)) + (44.23883762033049 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.20988449057976866 = (371.1967131260907 - 293.2882800867366) / 371.1967131260907 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDeepUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 33.66 ms 24.84 ms 0.74 0.08 True
[WriteDeepUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 33.61 ms 24.65 ms 0.73 0.10 False
[WriteDeepUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 33.63 ms 24.94 ms 0.74 0.10 False
[WriteDeepUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 41.70 ms 24.90 ms 0.60 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.84050285714286 < 31.930605430555552. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 30.56006053832709 (T) = (0 -24965227.611599512) / Math.Sqrt((1385508092062.6929 / (299)) + (409195117891.3576 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.26468617038925535 = (33951799.36275023 - 24965227.611599512) / 33951799.36275023 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.652508333333333 < 31.749254125. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 86.14764083872852 (T) = (0 -24650952.537284147) / Math.Sqrt((1644911949124.4263 / (299)) + (39960681877.020836 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2700782890684235 = (33772050.02687603 - 24650952.537284147) / 33772050.02687603 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.93563076923077 < 31.942487630952378. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 53.569019587267164 (T) = (0 -24960908.362986214) / Math.Sqrt((2156944576919.9011 / (299)) + (151145039897.96844 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2669944026804001 = (34052820.95288412 - 24960908.362986214) / 34052820.95288412 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.89760142857143 < 31.672107791666665. IsChangePoint: Marked as a change because one of 3/12/2022 1:22:26 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 99.73590469135111 (T) = (0 -24628992.36002093) / Math.Sqrt((1115991913721.65 / (299)) + (31713151723.361942 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2690612672810998 = (33695016.09034665 - 24628992.36002093) / 33695016.09034665 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Clear - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Int32).Clear(Size%3a%20512).html>) 36.05 ns 29.76 ns 0.83 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.Clear(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.76225473197059 < 34.25247310872807. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 50.46933087847018 (T) = (0 -29.802643544195355) / Math.Sqrt((1.674060473926128 / (299)) + (0.07158700932654698 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1756220173687873 = (36.151673349005044 - 29.802643544195355) / 36.151673349005044 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes%3a%20100).html>) 89.65 ns 74.10 ns 0.83 0.07 False
[WriteByteArrayAsBase64_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes%3a%20100).html>) 86.65 ns 75.77 ns 0.87 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 74.09609087171118 < 81.58182134349995. IsChangePoint: Marked as a change because one of 3/30/2022 4:48:39 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 29.856370896357316 (T) = (0 -74.17394784371072) / Math.Sqrt((5.015808684283663 / (299)) + (0.9194664720318007 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13413813845880782 = (85.66487466220615 - 74.17394784371072) / 85.66487466220615 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 75.77210071062127 < 80.76386361114174. IsChangePoint: Marked as a change because one of 3/30/2022 1:01:49 AM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 33.44315461180921 (T) = (0 -75.06818386774262) / Math.Sqrt((4.800641336742727 / (299)) + (0.5873478071690194 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1234638172532356 = (85.64185409038635 - 75.06818386774262) / 85.64185409038635 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Clear - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).Clear(Size%3a%20512).html>) 17.62 ns 6.26 ns 0.36 0.23 True
[Fill - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).Fill(Size%3a%20512).html>) 14.85 ns 5.18 ns 0.35 0.05 False
[IndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).IndexOfValue(Size%3a%20512).html>) 9.56 ns 8.44 ns 0.88 0.13 False
[LastIndexOfValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Byte).LastIndexOfValue(Size%3a%20512).html>) 16.00 ns 11.60 ns 0.72 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.Clear(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.256767557185643 < 16.989375952994465. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 77.85678289682684 (T) = (0 -6.475314646446255) / Math.Sqrt((1.0848067669528825 / (299)) + (0.1370991451162791 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.6468833833200175 = (18.337609561757354 - 6.475314646446255) / 18.337609561757354 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.Fill(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.1828293182434075 < 14.06706599546068. IsChangePoint: Marked as a change because one of 4/5/2022 2:02:34 AM, 4/7/2022 3:14:09 PM, 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 83.74817427641123 (T) = (0 -5.224618925854352) / Math.Sqrt((1.0968634952754215 / (299)) + (0.07112846996654082 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.6533873534622637 = (15.07336497396248 - 5.224618925854352) / 15.07336497396248 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.437601399898021 < 9.088963429168635. IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/21/2022 5:54:30 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.421935152629587 (T) = (0 -8.434750075210063) / Math.Sqrt((0.21082312526645614 / (299)) + (6.096515679366481E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0795305477578026 = (9.163530690414134 - 8.434750075210063) / 9.163530690414134 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.LastIndexOfValue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.599024275732777 < 15.082946774411916. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/21/2022 3:06:05 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 16.139938761158046 (T) = (0 -11.623015925734947) / Math.Sqrt((4.298800729322196 / (299)) + (0.001973861897047322 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14440192233734594 = (13.584668116000232 - 11.623015925734947) / 13.584668116000232 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare 1e41844947bc68735ddbf3e6aec724f78f03bc2a
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Clear - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.Span(Char).Clear(Size%3a%20512).html>) 30.89 ns 19.61 ns 0.63 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.Clear(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.60638544392759 < 24.96049974933169. IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.99171269325093 (T) = (0 -20.113269118513227) / Math.Sqrt((1.4817038527481252 / (299)) + (0.557136632370833 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24946402939272289 = (26.798541184160815 - 20.113269118513227) / 26.798541184160815 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)